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Ms. Nancy Livingston 
State Office of Education 
250 E. 500 S. 
Salt Lake City, UT 84111 

Dear Ms. Livingston: 

As co-directors of the Utah Writing Project, we are submitting our 
final report which describes the development and field testing of a pilot 
writing assessment for Grades 3 and 8, This assessment was keyed to 
selected standards from the Utah Core Curriculum in Language Arts, 

The Utah Writing Project began this project in June 1987 and has 

car^ ^d out a variety of activities since that time. The Project has 

invo.yed over 50 teachers in the pilot assessment and has had 
participation from 16 school districts throughout the state. 

The Overview and Summary of Findings which begin the accompanying 
report will provide you with a synopsis of our activities. The full 
report provides a detailed accounting of the writing assessment and the 
results as well as a series of recoirmendations . 

We hope that our development and field-testing of materials will 
provide the State Office of Education with useful information as it looks 
ahead to planning a state-wide writing assessment in the near future. We 
would be most interested in discussing our findings and recommendations 
with staff of the State Office at their convenience. 

With this final report, we believe we have fulfilled our obligations 
under the contract assigned to us by the State Office in June of 1987. We 
hope this report will receive careful attention and will lead to further 
discussions concerning writing assessment in the state of Utah. 

Sincerely, 




Charles R, Duke 

Co-director, Utah Writing Project 




William Strong ' 
Co-director, Utah Writing Project 
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I. OVERVIEW 

The pilot test of the Utah Assessment of Writing/Language skills 
represents an effort to develop a model for state-wide assessment of student 
writing/language skills in conjunction with Utah's Core Curriculum in 
English/Language Arts (UCCLA). The specific goals of this assessment were as 
follows: 

1. To establish procedures for administering and scoring writing samples 

2. To develop and test a variety of writing tasks ("prompts") and scoring 
rubrics to determine their effectiveness 

3. To establish procedures for reporting the data collected by such 
testing 

Background . 

The Utah State Office of Education (USOE) contracted with the Utah 
Writing Project (UWP), situated at Utah State University, to conduct a pilot 
project to test procedures for collecting baseline data on writing/ language 
skills at grades 3 and 8. Since the overall aim of this assessment was to test 
the validity of various standards set by the Core Curriculum, implemented in 
1985, all writing tasks were developed in relation to core standards. 

The UWP was charged with development of test instruments, field testing, 
and analysis/interpretation of the data. To provide an appropriate assessment 
of writing at grades 3 and 8 keyed to the Utah Core Curriculum in Language Arts 
(UCCLA), UWP had to develop materials that would be responsive to the 
variations in ability found at the two grade levels and distinguish between 
writing that fulfilled a particular task and that which did not. 

The assumption that unde'^lies the UWP approach is that every writing task 
represents an effort to accomplish a certain goal. If the writing task is 



carefully defined, some writing strategies will ensure success in accomplishing 
the task; others will not. For example, a writing task i.,at requests 
information in letter form assumes that certain strategies result in the 
requested information being presented clearly and accurately; a letter that 
does not accomplish this task, no matter how well written, is inappropriate. 

To develop assessment materials, it was necessary to identify certain 
goals--in this case, the standards from the UCCLA-and create "writing prompts" 
or tasks that defined the goals as accurately as possible. An exercise in 
which students write about anything of interest would result in a writing 
sample, but would yield little information regarding the Core Standards. 

Because this project was designed to gather information useful for a large 
scale writing assessment, the UWP developed writing tasks which vary in the 
amount of information they provide and the degree of choice they permit the 
respondents. In developing the tasks, the UWP followed six steps. 

Step 1. Fourteen teachers who had been previously trained by the UWP in a 
basic summer institute were identified. These teachers represented both 
elementary and secondary school levels. During the summer of 1987, these 
teachers attended a three week institute at Utah State University during 
which the following occurred: (1) development of writing prompts keyed to 
selected standards in the Core for grades 3 and 8; (2) development of 
scoring procedures; (3) mini-pilot testing of selected prompts and scoring 
rubrics at grades 3 and 8. 

Step 2. Sixteen school districts were selected at random according to 
size and geographical location. Superintendents of each district received 
letters inviting them to participate. All agreed to do so, with two 
schools, an elementary and a secondary or junior high school chosen from 



each district. Just before testing, a junior high from one of these 
districts could not .wticipate; therefore, a junior high from another 
district was added. Principals at each school made the selection of two 
third grade classes or two eighth grade classes to serve as the test 
population. (See Appendices A and B for a list of districts and schools 
participating in the study). 

Step 3. Procedures for administering the test were developed and a 
schedule established for test administration (see Appendix for sample test 
administration materials). Testing of approximately 1700 students 
oc.urred during the first two weeks of October, 1987. 
Step 4. Holistic scoring sessions involving Utah public school teachers 
from both elementary and secondary levels were held on three different 
occasions. Approximately 53 teachers participated in the scoring of 
writing samples. 

Step 5. Data from the scoring sessions were analyzed by computer. 
Step 6. This data was incorporated into the final report. 

Organization of this Report 

The first three sections of the following report contain a summary of the 
design and methodology of the assessment. The remaining sections present the 
results of the assessment with accompanying discussion. 



SUMMARY OF FINDINGS 



1. The process for development of writing prompts (analysis of UCCLA 
standards, mini-field testing, rubric development, and full field-testing) 
yielded useful materials for large-scale assessment. 

2. The materials developed to support the pilot test (ger.eral directions for 
teachers, scripted prewriting scenarios, letters to administrators, etc.) 
assisted in the assessment process. 

3. The process of prompt development and holistic scoring was viewed 
positively by teachers as an important and useful activity in their 
professional growth. 

4. The scoring of 1725 writing samples yielded high levels of inter-rater 
agreements-coefficients of -96 at grade 3 and .93 at grade 8; inter-rater 
agreement on one eighth grade prompt reached 100 percent. 

5. All writing prompts produced an acceptable range of writing. 

6. At the third grade level. Prompt 2 (Possession) emerged as the most 
effective of the four prompts tested, followed by Prompt 3 (Unusual Day); 
the least effective was Prompt 4 (Scary Experience), 

7. At the eighth grade level. Prompt 8 (Snakes) was most effective of the 
four prompts tested, followed by Prompt 6 (Possession) ; the least 
effective was Prompt 7 (Music). 

8. Prewriting at grade 3 had mixed results, probably because students have 
had little experience with prewriting and/or felt little need to develop a 
lengthy text after making prewriting notes. 

9. An effective prewriting sequence can be administered as part of a 45 
minute test period for eighth graders. 

10. The two prewriting conditions at grade 8 (Prompt 6, Possession) clearly 
had a positive impact on student writing when compared to two conditions 
without prewriting. 

11. The average mean for male third graders was 3.32, with a standard 
de»'iation of 1.59; for females, the average was 3.84, with a standard 
deviation of 1,74. At the upper score ranges (score of 6 or above) 
females showed a higher percentage of achievement (18 percent compared to 
8 percent) . 

12. The average mean for male eighth graders was 3.57, with a standard 
deviation of 1.49: fc- females, the average was 4.46, with a standard 
deviation of 1.59. At the upper score ranges (score of 6 or above) 
females showed a higher percentage of achievement (25 percent compared to 
12 percent). 
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13. A total of 6.6 percent of grade 3 papers we»-e unscorable; a total of 1.7 
percent of grade 8 papers were unscorable. 

14. Third graders, at least at the beginning of the year, are not able lo 
handle independent reading of writing task?. 

15. Students who scored highest on the assessment wrote twice as many words as 
those who scored at the lowest point. 

16. Error rates in student writing tend to increase as qualitative scores 
decrease. 

17. Third graders and eighth graders exhibit low rates of error for sentence 
fragments and subject/verb agreement but higher rates for spelling. 

18. Third graders, in general, did not use cursive writing in responding to 
the writing tasks. 

19. For all score points in the assessment, error rates decline in virtually 
all areas over five years of schooling, 

20. The system developed for coding of information, when used in conjunction 
with VAX programs, yields charts that will be useful in reports to 
districts. 



II. DESIGN 

After reviewing writing assessments conducted by other states, (nost 
notably the one in California), and work of the National Assessment of 
Fducational Progress (Beaton, 1987), the UWP co-directcrs decided to test 
multiple prompts under varying conditions to deterr ne which proiripts and which 
conditions provided the most effective assessment of writing. Participants in 
the UWP Advanced Summer Institute devised a number of differsnt proiipts based 
on certain standards in the Core Curriculum. At grade 3, ^our parallel prompts 
were developed for a single standard (narrative writing). At grade 8, four 
prompts were developed for four different standards (two related to informative 
writing, one in narrative writing, and one in persuasive writing). 

The grade 8 assessment differed from tho grade 3 assessment in that eighth 
grade cl;»sses were given multiple prompts; that is, instead of a class 
receiving one prompt, it received four; these were distributed on a random 
basis among students in the class. This was done to ensure that a range of 
students responded to each prompt and to test the feasibility of conducting 
such an assessment. With multiple prompts there is less likelihood that 
teachers and students will "prep" for one kind of w.-iting. 
GRADE 3 STANDARn . 

In the Core Standards in Language Arts for Grade 3, the "Writing Standard" 
(4030-07) called for "Students . . . [to] express ideas and experiences in 
written form." Under this standard were listed the following objectives: 
4030-0701 Generate and organize ideas for writing 
4030-0702 Write personal experiences, stories, poetry, etc. 
4030-0705 Use capital letters and terminal punctuation 

6 
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Since narrative is a common element in third grade student writing 
(Graves, 1983; Mosenthal et al., 1983), the decision was made to emphasize 
narrative along with personal experience. In addition, the following 
objectives taken from other standards were deemed appropriate for grade 3 
writing assessment: 

Standard 4030-05: The students will correctly spell words needed to record 
Ideas and experiances. 
4030-0503 Write words and sentences with correct punctuation and 
spelling 

4030-0504 Spell homonyms and contractions correctly 

4030-0506 Show understanding of spelling words by telling what 

they mean or using them in a sentence 
Standard 4030-06: The students wiT Maintain manuscript skills and learn 

to write cursive. 

4030-0601 Use correct formation of all upper and lower case 
letters and numbers in cursive 
Grade 3 Test Tt.Pirn: . 

Although the decision was made to emphasize narrative, the pilot test was 
designed to assess the effectiveness of more than one prompt. Four prompts 
were selected for this purpose and appear in Exhibit 1. 
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Exhibit 1: Writing Prompts for Third Grade 



Prompt 1 Having fm at a Favorite Place 

Background 

Most people have hej a fun experience at a favorite place. For you, it might 
be buildinq a hut in the backyard, playing ball in a neighborhood park, 
roasting marshmallows at a campground, or riding the roller coaster at 
Disneyland. Think about what you really like to do at your favorite place. 

Directions 

Use this page to plan your writing. Then, beginning on the next page, tell a 
story about a time when you had fun at your favorite place. The reader would 
like to know what this place is like, so describe it. Tell what you did, how 
you felt, and why it was fun to you. 



Prompt 2 HoM I Got Ny Favorite Possession 

Background 

Like most people, you probably own something that is very special, it might be 
a pet, a toy, a book, or something you wear. Maybe this possession is special 
because of the person who gave it to you or because you earned it yoursel-f. 
Think about the day you got a favorite possession and why you like it. 

Directions 

Use this page to plan your writing. Beginning on the next page, tell a story 
about when you got your favorite possession. The reader would like to know 
what it looks like, so describe it. Tell what happened when you got it, how 
you felt, and why it is imoortant to you. 



Prompt 3 An Unusual D^y at School 

Background 

All of us have had an unusual day at school. For you, it might have been a 
special part in a school program, a dress-up day, or a substitute teacher. 
Maybe you won an award or celebrated a holiday. Think about an unusual day at 
school and what made it a day you still remember. 

Di rections 

Use this page to plan your writing. Beginning on the next pago, tell a story 
about an unusual day at school. The reader would like to know what this day 
was like, so describe it. Tell what happened, how you felt, and why you 
remember this unusual day at school. 
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Prompt 4 



A Scary Experience 



1 



Background 

Most people have had a scary experience. Maybe they were frightened by a 
snake, or big dog or even a bee. Maybe they were scared by a bully at school. 
Maybe they were lost in a mall or frightened by an audience. Think about a 
scary experience you have had and why you still remember it. 

Directions 

Use this page to plan your writing. Beginning on the next page, tell a story 
about a time you were scared. The reader wants to know where you were and what 
made you feel scared. Tell what happened, how you felt, and why you still 
remember this scary experience. 



Conditions . 

An effort also was made to determine under what conditions the best writing 
could be expected to occur. With that in mind, four conditions were identified: 

1. Oral reading with teacher-directed prewriting 

2. Oral reading without teacher-directed prewriting 

3. Silent reading with teacher-directed prewriting 

4. Silent reading without teacher-directed prewriting 

A mini -pilot test prior to the fur field test demonstrated that third 
graders were not ready to handle Conditions 3 and 4; for the third grade level, 
therefore, these were deleted and only Conditions 1 and 2 appeared in the full 
field test for third grade. 

The directions for Condition 1 were basically "-he same for all prompts. 
Students and teachers were given a prewriting sequence to follow. students 
t?c9ived a sheet on which appeared the prompt, prewriting directions, and space tor 
prewritipq. Teachers eceived explicit directions on how to use the prewriting 
sequence and were asked to follow these directions as carefully as possible. (See 
Exhibit 2 for a sample). 

9 
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Exhibit 2: Student Prewriting Prompt Sheet 



3-41 

UTAH WRITING ASSESSHENT -- PILOT TEST 10/87 



A Scary Experience 

Background 

Most people have had a scary experience. Maybe they were frightened by a 
snake, or big dog or even a bee. Maybe they were scared by a bully at school. 
Maybe they were lost in a mall or frightened by an audience. Think about a 
scary experience you have had and why you still remember it. 

1. Write down a place where you were scared. 



2. What made this experience scary? Write this down. 



3. What words describe this scary experience? List them. 



Directions 

Use this page to plan your writing. Beginning on the next page, tell a story 
about a. time you were scared. The reader wants to know where you were and what 
made you feel scared. Tell what happened, how you felt, and why you still 
remember this scary experience. 
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Oral reading with teacher-directed prpwriting 

Read "Background" aloud to the class as students follow along. Direct 
them to do the three prewriting steps that follow the "Background." 
Invite a few students to share aloud some of their words and ideas from 
the three steps; acknowledge these ideas positively. Then read 
"Directions" aloud to the class and tell students to begin their planning 
and writing. (Total time should be about fifteen minutes.) 
GRADE 8 . 

A review of the Core Standards for Grade 8 reveals, as one might expect, 
more variety in writing modes and in objectives. In the interest of efficiency 
and in an effort to retain a sufficient population from which some preliminary 
findings might be dr^wn, only four modes of writing were selected: the 
informative report, persuasive speech, the response to a request for 
information, and narrative. Each of the modes had corresponding objectives to 
be measured. Because multiple modes were tested at this grade level, testing 
was done for three of the four modes only under condition 4--silent reading 
without teacher direction. One prompt--the na.rrative--was done under all four 
conditions (oral/silent reading; with/without prewriting). 

THE INFORMATIVE REPORT 

The informative report calls for the student to use controlled data from 
one or more sources to provide an explanation on some subject. The following 
standards and objectives from ihe UCCLA address this mode: 

Standard 4080-08! Students mIII conpose, . . . In . . . written language. 
Informative reports based on a variety of sources. 

Of the ten objectives listed under this standard, the following five were 
considered applicable to the testing situation: 

11 
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4080-0801 Select an audience and a manageable topic that will 

interest the audience 
4080-0804 Synthesize information accurately 

4080-0805 Use a logical system of organization; that is, either 
general to specific (deductive) or specific to general 
(inductive) 

4080-0808 Use the entire composing process; that is, prewriting, 

drafting, revising, and editing 
4080-0810 Use vocabulary appropriate to the selected audience 
Standard 4080-09: Students will produce, through the editing process, a 

well organized and informative final draft that conforms 

to standard English. 
4080-0901 Use appropriate sentence structure to convey ideas. Use 

short and simple sentences for main ideas, longei and 

m^e complicated sentence structure for supporting 

details. 

4080-0902 Combine sentences when they share the same subject or 

verb and do not present major new ideas. 
4080-0903 Avoid obvious redundancies 

4080-0904 Define unfamiliar or technical vocabulary for the 
audience. 

4080-0905 Begin new paragraphs when the subject matter 

significantly changes 
4080-0906 Use standard spelling 
The Prompt and Tes t Condition . The prompt for the Informative report uses 
what is called "controlled data" as a basis for the writing. Such a prompt 

12 



provided students with information that they used to develop a written 
response. The situation was a school assembly on snakes which was to be 
reported on in the school newspaper. See Exhibit 3 for the prompt. The prompt 
was tested under only Condition 4--silent reading without prewriting direction. 
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Exhibit 3: Grade 8 Writing Prompt for Report Writing 



Writing Situation 

John Burton, an expert on snakes, presented an assembly in your school 
auditorium. He brought with him a live ^'^ttlesnake, an Indian python, and a 
king cobra. You are on the newspaper stafi * y^ur school and have been asked 
to write a news story about the assembly. 

Writing Directions 

Write a news story about the assembly for your ^hool newspaper. Assume 
you have taken the following notes. Select the infori..a m that you can use to 
write an effective news story. Remember that the fir naragraph of a n^^. 
story contains the 5 W*s (who, what, when, where, and wuy.) Organize your 
remaining information so that it is interesting and easy for your readers to 
understand. 

YOUR NOTES 



Diamond Rattlesnake 
named Elvis 
has long hollow fangs 
is most dangerous rattler 

in U.S. 
rests with body coiled 
has hollow teeth like 

hypodermic needles 
raises upper third of body 

into S*shape when angry 
stabs prey 

warns of danger with rattle 
swallows food whole 
lives in U.S. 
strikes full length of 

S- shape 
gives birth to live babies 



Python 
named Crackin 
8 ft. from India 
held by 14 members of 

audience 
lays eggs 
is sluggish, lazy 
is found near water 
is not poisonous 
kills by constricting 
large pythons kill goats, pigs, and deer 



King Cobr^ 

named Tinkerbell 

is 3 ft. snake from India 

strikes slowly 

attacks without being angered 
is hooded 

bites and chews with short fangs 
as venom runs into wound 

has deadly bite with no antidote 

was shedding and was irritable 
and nervous 

doesn*t move to snake charmers' 
music. 

Sways to charmers' movement 
king cobra kills 25,000 Indian 

people per year 
large king cobras have enough 
venom to kill hundreds of 
people 



python in Malaysia ate 3 14 
year old boy 
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PERSUASION 

Because the Core calls for both oral and written skills in persuasion and 
the standards address both, it was necessary to eliminate the oral skills and 
focus only on the written objectives under the appropriate standards. These 
appear below: 

Standard 4080-11: Students will clearlj reflect and support their ideas in 
persuasive papers .... 

4080-1101 Choose an issue about which the student has strong 
feelings and is willing to become knowledgeable 

4080-1112 Select an audience and determine what they will likely 
think about the chosen issue 

4080-1103 State the opinion clearly 

4080-1104 Support the opinion with examples, facts, or other 
evidence 

4080-1105 Use the entire composing process; that is, prewriting, 

drafting, revising, and editing 
4080-1107 Use words that persuade most effectively 
Standard 4080-12: The students will produce, through the editing process, 
persuasive final products that confon to standard 
English. 

4080-1201 Begin a new paragraph each time there is a significant 
shift in the reasons for the opinion or the support for 
one of the reasons 

•4080-1202 Make sure subjects and verbs agree in number . . . 

4080-1203 Use standard spelling 
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fhe Prompt and Test Condition . The prompt was written to create a context 
in which written skills might prepare the student for oral speaking on a 
controversial topic. The prompt was tested under only Condition 4--silent 
reading without prewriting direction. See Exhibit 4 for the - -npt. 



Exhibit 4: Grade 8 Writing Prompt for Persuasion 



Writing Situation 

Parents and students at your school are divided in their opinions about 
the effect of rock music on students. Some want to eliminate all rock music on 
radio stations and at school dances. Others don't view rock music as a 
problem. 

Because the issue is one that has caused great interest among people in 
your community, the local radio station has scheduled a public meeting where 
people can speak about this issue. 

Writing Directions: 

I 

Decide which side of this issue you favor. Write a speech designed to 
persuade others to agree with you. Your speech should include personal 
experience and/or other evidence to support your position. Your audience will 
consist of both parents and other students. 



16 



INFORMATIVE RESPONSE TO REQUEST FOR INFQRMATTnN 

Although the Core Curriculum standard for informative responses also 
covers requests for information, only the informative response was used. Those 
objectives which addressed the response were as follows: 

Standard 4090-14: Student will compose . . . informative respon'es to 
requests. 

4080-1407 Identify who wrote the request and why 
4030-1408 Respond clearly to the request 
4080-1409 Use the appropriate letter format 

4080-1410 Use the entire composing process; prewriting, drafting, 

revising and editing 
4080-1411 Use vocabulary appropriate for the intended audience. 
Standard 4080-15; Students will produce, through the editing process, 

final drafts of ... an informative response that 

conforms to standard English. 
4080-1501 Define unfamiliar terms, using appositives wherever 

possible 

4080-1502 Punctuate appositives correctly 

4080-1504 Begin a new paragraph each time the information in the 

explanation significantly changes 
4080-1505 Use specific nouns and active verbs 
4080-1506 Avoid fragments and run-on sentences 
4080-1507 Use sentence variety to improve readability 
4080-1508 Use standard spelling 
The Prompt and Test r.ondition. The prompt was selected to elicit as much 
detail as possible as well as a sense of voice in the writer's response. 

17 
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Rather than write a formal business response, students were invited to respond 
to a peer's request for information. The prompt was tested only under 
Condition 4--silent reading without prewriting direction. See Exhibit 5 for 
prompt. 



Exhibit 5: Grade 8 Writing Prompt for Response to Request for Information 



Writing Situation 

Assume that your principal has received the following letter with a note asking 
that it be passed on to an eighth grader for a response. The letter has now 
been given to you. \ 

Dear Eighth Grader, 

I will be Tioving to your area and will be in the eighth grade 
at your school next year. Please write me a letter telling me 
about your school . 

I would especially like to know about teachers, classes, 
activities, and anything else that will help me fit in to the 
school . 

Write soon. 

Sincerely, 
Kelly Green 



Writing Directions: 

Write a letter to Kelly that tells about your school. Use your own mailing 
address for the return address so that if Kelly wants to write for more 
information, you will receive the letter. The principal has Kelly's address and 
will see that your letter is sent. Since Kelly has never visited your school, 
provide as much helpful information in your letter as you can. Be certain to 
include the information Kelly has requested. Write only on one side of the 
page. 
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NARRATIVE 

Core standards address nan we writino at the third grade and eighth 
grade levels. While standards differ slightly, focusing on observed experience 
at the eighth grade rather than direct personal experience, many of the 
objectives under the eighth grade narrative standards are similar to those at 
the third grade. Hence, an effort was made to test at th') eighth grade level 
one of the narrative prompts used at the third grade level. The eighth grade 
objectives to be measured were as follows: 

Standard 4080-05 : Students will conpose narratives of observed experience 
with an implied simple theme. 
4080-0501 Select an audience and an interesting topic 
4080-0502 Develop a logical story line (plot) 
4080-0503 Develop characters by showing specific details of 

appearance, action, and speech 
4080-0504 Use the entire composing process; that is, prewriting, 

drafting, revising, and editing 
4080-0505 Use precise vocabulary to create the ueslred dominant 
impression and to relate an Important theme 
Standard 4080-06 : Students Mill produce, through the editing process, 
final drafts of observed experience that conform to 
standard English. 

4080-0601 Use clear and specific language that reveals the 

character and situation 
4080-0602 Use adjectives and adverbs correctly . . . 
4080-0603 Make subjects and verbs agree In number 
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4080-0604 



Create and correctly punctuate more sophisticated 



sentence structure 



4080-060S 



Avoid fragments and run-on sentences 



4080-0607 



Begin new paragraphs each time the action, setting, 
idea, or speaker changes 



4080-0608 



Use standard spelling 



The Prompt and Test Conditions: Prompt 2 (See Exhibit 6) from the third 
grade assessment was selected for this comparison; the features of the prompt, 
although concentrating on an object rather than people, fit most of the 
elements called for in the observed experience narrative of the eighth grade. 
The prompt was tested under four conditions. (See Appendix C for explanations 
of each of the conditions). 

1. Oral reading with teacher-directed prewriting 

2. Oral reading without teacher-directed prewriting 

3. Silent reading with teacher-directed prewriting 

4. iilent reading without teacher-directed prewriting 

Exhibit 6: Writing Prompt for Narrative at Grades 3 and 8. 



Like most people, you probably own something that is very special. It might be 
a pet, a toy, a book, or something you wear. Maybe this possession is special 
because of the person who gave it to you or because you earned it yourself. 
Think about the day you got a favorite possession and why you like it. 



Use this page to plan your writing. Beginning on the next page, tell a story 

about when you got your favorite possession. The reader would like to know 

what it looks like, so Describe it. Tell what happened when you got it, how 
you felt, and why it is important to you. 



HoM I Got Ky Favorite Possession 



Background 



PIrggtiQns 
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III. DATA COLLECTION AND HANDLING 

Field Contact 

UWP initiated contact with school and district personnel in early 
September of 1987 to alert them to the pilot assessment and to provide 
information on the local phases of the assessment. Sixteen districts and 
thirty-two schools were selected for the pilot sample. 

Superintendents of sampled districts received a letter outlining the 
schedule of events and inviting par. .cipation; specific schools were named that 
had been targeted for assessment. (See Appendix D for sample letter). 

Principals of the selected schools received a letter requesting their 
cooperation, describing t^z responsibilities, and asking them to identify two 
classes to be used for the testing. Principals were asked to respond with the 
number of students in each class that would be tested. In an effort to 
minimize disruption of school schedules, principals were given a small "window" 
of two or three days in which to carry out the assessment. (See Appendix E for 
sample latter). 

The UWP staff made every effort to describe in detail the purpose of the 
assessment and the responsibilities of principals and teachers who were 
involved. Throughout the contact period, UW encouraged school personnel to 
call with any questions or concerns relative to the assessment. Teachers who 
administered the test were asked to fill out a survey to provide information on 
any unusual problems encountered in the testing. (See Appendix for D survey). 

Test Administration 

Test packets with instructions were mailed first class to t ch school at 
least a week in advance of the test date. Principals were asked to provide the 
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teachers who would be administering the test with background information (See 
Appendix C) but were to retain the actual writing prompts until the test dates. 
Testing took place on October 1-2, 1987 in all but three schools; at these 
locations, testing occurred within a week of the scheduled '<ate. 

Regular classroom teachers conducted the test session. Principals were 
asked to visit the testing sessions and to maka notes on any problems or 
concerns which surfaced. Students had approximately 45 minutes for thr 
writing; in those instances where prewriting occurred, the prewriting activity 
was included in the 45 minutes. More than 1700 students enrolled in 32 schools 
took the tests. A breakdown of the test population appears in Table 1 below. 



TABLE 1 Smury of Frequencies 


Total Test Population: 1725 




number of 3rd graders tested: 863 




number of 8th graders tested: 862 




number of males tested at third grade level: 


461 


number of females tested at third grade level: 


400 


number of males tested at eighth grade level: 


373 


number of females tested at eighth grade level: 


488 



Coding. Scoring, and D ata Processing 

All cover sheets for each writing sample were subjected to :.n in-house 
edit by UWP staff, including (1) a check on the coding of student, gender, 
district, and school identification information; (2) an edit for stray marks on 
the cover sheet; (3) a totaling of the holistic :cores on each cover sheet. 
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Written responses received hoi istic scores--see discussion of scoring 
procedures—which were totaled on each writing samplers cover sheet. These 
scores were then keypunched, verified, and listed on a data tape which was then 
converted to diskette as well as to a VAX account. 

The data tape contained one record for each student completing the writing 
prompt. This record included (1) a numerical identification code which 
permitted the rematching of each student record to its original test document 
should that prove necessary (2) gender code; (3) district code; (4) school 
code; (5) grade level code; (6) prompt code; (7) condition code; (8) holistic 
score code; (9) third reading code. 

Prior to data processing, the data tape was scanned for invalid entries 
and lodated where appropriate. All response data on the data tape were entered 
in raw -score form and converted to the percentage form as needed. 

Reports generated from the data included the following: 

1. Performance by all third grade students 

2. Performance by third grade students on each prompt according to 
condition 

3. Performance by gender of third grade student on each prompt and 
condition 

4. Performance by all eighth grade students 

5. Performance by eighth grade students on each prompt according to 
condition 

6. Performance by gender of eighth grade student on each prompt and 
condition 

7. Number of third readings by prompt and condition at each grade 
level 
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The Mechanics of Test Administration and Scoring 

The mechanics of developing and administering the pilot test for writing 
assessment required that UWP deal with coding, teacher direction, and holistic 
scoring, which are described as follows: 

Coding . 

One of the initial tasks faced by the UWP staff was determining how to 
code the prompts to protect the anonymity of the students and schools involved. 
There was also the need to streamline the process so that valuable time would 
not be lost during the actual testing situation. The decision was made to pre- 
code all the writing prompt sheets so that prompt number, condition and grade 
level would not have to be entered by students. Therefore, in the upper right 
hand corner of each prompt sheet were placed three digits; for example, 3-4-1 
would indicate third grade, prompt 4, condition 1. This coding proved 
invaluable when it came time to send the pronpts out to the schools, since 
sorting the sheets would have been time-consuming without this prior 
identification. It also proved helpful when sorting the papers for holistic 
scoring sessions. 

In addition to the coding of the prompt, condition, and grade level, 
identification of each student, the student's gender, the district, and the 
school were needed. These were to be filled in by the student according to 
teacher direction. To avoid duplication of student identification numbers, 
teachers w^re instructed to have students use the last four digits of their 
telephone numbers. Although apparently some third graders did not know their 
telephone numbers, this practice worked fairly well. The coding for gender was 
done by using "T for males and "2" for females. Again, in some instances, 
students apparently got confused and put M or r in the gender slot; however, 
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this was not a major fact-". District and school codes were supplied to the 
teachers administering the tests and students placed those in the appropriate 
slots on the prompt sheet. A typical entry, then, would look like the 
following: 04cO 2 15 30. 

Ths only complication arose at the third grade level where some teachers 
reported that coding the prompt sheet took students much longer than 
anticipated, and it was suggested that perhaps some pre-coding, at least at 
that grade level, ought to be done prior to the testing session (See Appendix K 
for sample of coding). 

Teacher Directions . 

Efforts were made to provide teachers with as much information as 
possible. All test materials were delivered to the test site at least a week 
in advance, and teachers were notified by their principals that the materials 
had arrived. Included with the actual tests were suggestions to the teachers 
about how to prepare students for the testing in advance in order to minimize 
test anxiety; teachers were asked to develop a master list of student 
identification numbers so that if a student forgot his or her ID number the day 
of the test, the number could be supplied quickly. 

For the test itself, the administration directions were quite specific and 
followed a step-by-step procedure which, based on feedback from classroom 
teachers, seemed to minimize the confusion and provided a useful reference when 
questions arose. ,he UWP received very few telephone calls concerning test 
procedures although teachers were encouraged to call if questions arose. (See 
Appendix F for a summary of comments by teachers about the test and test 
procedures). 
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Holistic Scoring Sessions . 

Once the writing samples had been completed, the UWP staff was faced with 
scoring the 1725 papers. Originally the staff had anticipated using 20 
classroom teachers from both elementary and secondary levels to score the 
papers in two consecutive six hour scoring sessions. Because of scheduling 
conflicts, the number of scorers in the original sessions was reduced to 14 and 
the scoring time shrank to 10 hours instead of the original 12. Subsequent 
scoring sessions were held on three other occasions and ranged from an hour and 
half to two hours and involved anywhere from 6 to 25 teachers at a session. 

Prior to the scoring sessions, papers were sorted by prompt; no attempt 
was made to sort by condition; however, sets of papers from districts were 
broken up to provide a more random distribution of students for scoring 
purposes. 

Scoring sessions themselves were carefully structured. It is most 
important that a careful screening of papers be done prior to any scoring so 
that sets of sample papers representing the different score categories be 
identified, Myers (1980) and White (1987) offer useful guidelines for 
conducting effective scoring sessions. In the large scale scoring sessions, 
approximately an hour and a half was used to go through the sample sets of 
papers and achieve consensus on rankings; third grade papers, because of their 
brevity, could be scored more rapidly than eighth grade papers. It must be 
remembered, however, that each paper must be read by two readers so this factor 
must be figured into the time allotment. (See Appendix G for sample 
instructions to scorers). 
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IV. SCORING OF WRITING SAMPLES 

Description of the scoring method . 

Each of the four writing samples at each grade level was scored using the 
holistic scoring method. Holistic scoring is an impressionistic scoring 
process that allows ranking of writing samples on the basis of their overal 1 
quality. In the process, scorers are trained to understand general features of 
writing that determine levels of achievement on an ordinal scale. The major 
assumption of holistic scoring is that writing quality should be judged on its 
whole presentation, rather than individual qualities. Therefore, the "whole" 
of a piece of writing is assumed to be greater than the sum of its parts. 

Large scale writing assessments (see Myers, 1980; McCaig, 1981; White, 
1985; Breland, 1987) have adopted holistic scoring. With training, teachers 
recognize and agree upon common elements of effective writing and can rank 
pieces of writing with a high degree of agreement. Duke (1985) and Mellon 
(1975) report inter-rater reliability correlations as high as .70 to .80 when 
the holistic approach is used correctly. 

The holistic scoring procedure . 

Before any scoring of writing samples occurs, scoring criteria are 
est blished for each writing prompt in the form of a rubric. Initial scoring 
rubrics were developed by the UWP summer institute participants after reading a 
sampling of papers written to several of the prompts. These scoring rubrics 
were further refined by UWP staff based on a sampling of papers taken from the 
actual assessment. For each prompt, a set of "range finders" was identified. 
Range-finders are papers which represent a range of achievement demonstrated by 
a total set of papers. Each range-finder is assigned a score from 1 to 4, with 
1 representing a weak paper and 4 representing a superior paper. The 1-4 
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scoring range is commonly used (Myers, 1980) to avoid having papers "dumped" in 
a middle rating. 

Training sessions for readers are held prior to actual scoring. Readers 
are asked to read a set of papers and assign each one a score from 1 to 4; 
readers then are asked to identify their rankings of each paper and these 
rankings are charted so that everyone can see the distribution; head readers 
then lead a discussion of rankings, seeking clarification, checking the scoring 
rubric against the rankings, etc. Usually i second readinc; is done with 
another set of papers and the charting process is repeated. If readers are 
achieving consensus on their rankings, the actual scoring can begin. If there 
is still 3 wide range in the rankings, further discussion and reading of papers 
may have to occur. Such training sessions take between one to two hours for 
each prompt used. For a specific formula for calculating the time involved in 
scoring papers hoi istically, see Myers (1980). 

During actual scoring sessions, each of two readers scores a piece of 
writing by reading it through relatively quickly and assigning it a number from 
1 to 4. (See Appendix G for scoring direction;). The score received by the 
writer on a writing sample is the sum of the scores of the two readers; scores, 
therefore, can range from 2 (weakest) to 8 (strongest). 

The holistic method guarantees that the two scores assigned a particular 
writing sample will not differ by more than one point. In cases where two 
readers differ substantially, such as a 2 and a 4, the chief reader does a 
third reading and either agrees with one or the other of the readers or assigns 
a value between the two scores which then is doubled to produce the final 
score. 
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The process of adding the scores of two readers expands the scoring scale 
to provide for "borderline" cases. A paper which receives 3 from both scorers 
(for a total of 6) is likely to be stronger than a paper which receives a 2 and 
a 3 (for a total of 5). Likewise, a paper that receives a 4 and a 3 (for a 
total of 7) will be stronger than a 6 paper. 

Holistic readings done in the prescribed manner extend beyond an 
individual assessment. Such sessions help classroom practitioners become aware 
of the testing purposes and the handling of student work. These teachers 
return to their classrooms with a renewed sense of the validity of using actual 
.writing as a reliable test cf writing ability and a better understanding of how 
this reliability can be achieved. 
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V. RESULTS OF THE WRITING ASSESSMENT 

Interpretino the srnrp*: . 

Scoring rubrics are provided for each of the writing prompts. Since the 
writing prompts for the third grade were all narrative, one rubric was used to 
score all of the third grade samples. For the eighth grade, however, a 
different scoring rubric for each prompt was necessary, since each prompt 
called for a different kind of writing and had different objectives keyed to 
it. 

Rubrics provide "descriptors" that can be associated with papers in each 
scoring category. Although the descriptors identify characteristics common to 
papers with a particular score, not iH papers in a given category will 
necessarily contain all of the characteristics associated with that category. 

Papers which merely copy the assignment over, provide insufficient writing 
to be judged, miss the topic entirely, are illegible, or give no response are 
given a 0. A score of 0, however, is not to be viewed as worse than a 2. The 
0 indicates merely that a paper was not scorable and therefore no assessment 
could be made of the student's writing skills. Both readers must decide that a 
paper deserves a 0. Otherwise, a third reading is necessary. 

Examples of actual student papers which helped to define each of the four 
scoring categories should be read in order to understand the results which 
follow. In reading the papers and looking at the results, readers should keep 
in mind that all writing was done in first draft form and that no specific time 
was set aside for revising or proofreading. Essays have been reproduced 
exactly as received; specific names of individuals and schools are removed to 
preserve the anonymity of students and test sites. 
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Grade Three Resultc 

Eight hundred sixty-three (863) third graders provided the population for 
the third grade assessment. The third graders were asked to respond to four 
different prompts, all of which asked for the responses in narrative form. 
(Readers may wish to review the third grade prompts that appear in Section II 
of this report). Students were tested under one of two conditions: oral teacher 
direction with prewriting (Condition 1) or oral teacher direction without 
prewriting (Condition 2). 

Because all of the prompts called for narrative responses, only one 
scoring rubric was devised; the rubric was arranged to address all of the 
common elements identified in each of the prompts. Students had approximately 
45 minutes in which to complete any prewriting and a draft of their response to 
the writing prompt. The total number of writings by prompt and condition, with 
means and standard deviations for each scoring, appears in Table 2. 

On the following pages appear sample papers for each prompt used at the 
third grade level. Each paper represents a scoring category on the rubric for 
a particular prompt. Since the same rubric was used for all four prompts at 
this level, it appears first, followed by the representative papers and brief 
discussions of the results for each prompt. It should be noted that these 
papers are representative; they reflect a possible level of achievement at each 
score point; within each score category, of course, there is always some 
variability. The ratings which appear in the various tables throughout this 
section are based on the sum of two readers' scores; hence a score of 2 in the 
table represents two scores of 1 and 1; this pattern holds true throughout the 
ratings summaries, providing a scale of 0, meaning unscorable, to a maximum of 
8. 
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TABLE 2 Total Number of 3rd Grade Writings by Prompt and Condition 


Prompt Title 


1 Conditio 


n 2 Total 




Oral w/ 
Prewriting 


Oral w/o 
Prewriting 




Place (1) 


116 Count 
3.54 Mean 
1.44 Std. Dev. 


112 Count 
3.63 Mean 
1.75 Std. Dev. 


228 Count 
3.58 Mean 
1.65 Std. Dev. 


Possession (2) 


112 Count 
3.85 Mean 
1.56 Std. Dev. 


123 Count 
3.74 Mean 
1.65 Std. Dev. 


235 Count 
3.80 Mean 
1.60 Std. Dev. 


Day (3) 


99 Count 
3.09 Mean 
1.53 Std. Dev. 


103 Count 
4.14 Mean 
2.12 Std. Dev. 


202 Count 
3.62 Mean 
1.95 Std. Dev. 


Scary (4) 


99 Count 
3.53 Mean 
1.60 Std. Dev. 


99 Count 
3.10 Mean 
1.80 Std. Dev. 


198 Count 
3.31 Mean 
1.76 Std. Dev. 


Totals 


426 Count 
3.52 Mean 
1.50 Std. Dev. 


43? Count 
3.67 Mean 
1.86 Std. Dev. 


863 Count 
3.59 Mean 
1.74 Std. Dev. 
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PERSONAL' EXPERIENCE PROMPT 1-4 



OOWTEOT 


4 

Prcvldes rich 
descriptive back- 
cnmrvi of Gb'lects. 
places or people 
for e)^)erienoe 


3 

Provides descriptive 
background of 
obiects. nlaoes* or 
people 


2 

Name objects, places 
or people with little 

emphasizes context, 
drawing attention 
away from incident 


1 

Lists or names 
objects with no 

XmSS^L X^ L^VJI 1 


iNciDefr 


Narrates vivid 
aooount of personal 
involvanent in an 
experience (**&hous 
story") 


Narrates personal 
involvement in 
experience ("tells 
story*') 


Lists action with 
some narration of 
personal experienoa 


Has II^^Ia ov* no 

development of action 
or personal ex- 
perience 


SIGNIFICANCE 


Stranoilv enchaslzes 
vAiy experience was 
important 


Unlles or statues uhv 
experience was 
iivportant 


Gives llttiA -Irvlloji*- 

tlon eaqperlence 
was l]i|)Qrteait 


ijxves xxuuie or no 
indication of why 
e)qperienoe was 
i]i{X)rtant 


oohei^ce; 


DiEplays well- 
developed narrative 
structure (beginning, 
middle, end) ; or has 
thematic unll^ 


Displays narrative 
stnicture (beginning, 
middle, end) ; or has 
thematic unity 


Dlsplaya limited 
narrative structure 
(particularly with 
middle or end) ; or 
has little thantic 
unity 


Di^lays little or no 
development of 
narrative structure 
(beginning, middle, 
end) ; or laciks 
thematic unity 


OONVENTIONS 

38 


Has strong control 
over mechanics 


May have errors; 
these do not inter- 
fere with ocnimmica- 

tiCMl 


Has errors that may 
interfere with 
ocnnunicatiOT 


Has erroTB that 
interfere with 
ccTimonication 
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Sanple Response to IVoapt 1: 'Having Fun at a Favorite Place' 

Score: 1 

I like logoon becose it is fun and it has fun rides and it is scery anu 
has fun rides and it is exciting. 

Score: 2 

Once me and my family went to Disneyland. I like to ride the 
Materhorne, Thudur mountain, Dumbo, Space mountain and the submarine 
ride. Evrey morning we would ride the moirail to a resterunt in 
Disneyland and have Mickey Mouse pancakes and sometimes we would have 
beljun wafles and then we would go have diner at a restrunt or 
sometimes we would have diner at are hotel. After three days we would 
pack are stuff and drive about half an hour to another hotel and we 
would unpack are stuff and we would go th' beach for four days and then 
we would pack up and go home. 

Score: 3 

I like lagoon very much it has rides for every age espeshaly mine. I 
like going on tidale wave, colosus, screamer, the roller coaster, 
mousetrap and the logride. I dont like the new flying carpet any more 
because the first time I went on ;t wen I was on my way to our truck 
and I felt sick one minuter latter I stoped and waited for my dad to 
open the back of or truck I had to lay down in the back of the truk. 
When I got home I was asleep so my mom and dad had to wake me up when I 
got in the house I got in my pjs and brused my teeth and went to bed. 
the next morning as soon as I couV play I ran to my friends and told 
them abot lagoon, the next day I went to lagoon again. 

Score: 4 

Going to Pergatory is so much fun! Thare is a white slide and you go 
down it with sleds! I like the slide the best. Their Barb-que lunches 
are terriffic! They have steak, macaroni and mashed potatoes and soda. 
And for mi nature golf they have these objects and you have to smack 
them in there and the balls go out the other side! Their tennis they 
have heart shaped balls and net. And once I won a game against my dad! 
Their swimming pools are all on too of the motel! All of them! And 
their are 2 swimming pools and 3 hot tubs. And the inside of the 
motels are beautiful! And all of the beds are king size beds! It is 
the best place I've been to! 

Summary of Results for Prompt 1 

Prompt 1 invited students to identify a favorite place where they had fun. 
One of the key elements in this prompt was to determine if students cculd focus 
their writing on a particular time at this favorite place and describe it in 
such a way that readers would know why the student considered the place special 
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and what the fun woS that the student had there. No particular audience was 
specified, and no other format was suggested other than that of a "story" 
approach. Students, therefore, were drawing exclusively on personal experience 
to supply the information necessary in the response. 

Frequencies and percentages of scores. Two hundred twenty-eight (228) 
students responded to Prompt 1, which was tested under Condition 1 and 2. Tht 
mean for the responses was 3.58 with a standard deviation of 1.65. About 54 
percent of the students scored a 4 or above on this writing task, 11 percent 
achieved a ranking cf 6 or above while 1.8 percent achieved the highest ranking 
of 8. Table 3 below provides summaries or the percentages and frequenc js at 
each score point. 



TABLE 3 


GRADE: 3 


PROMPT; 1 


HOLISTIC SCORE 


VALUE 






CUM 


FREQUENCY 


PERCENT 


PERCENT 


0 


13 


5.7 


5.7 


2 


50 


21.9 


27.6 


3 


40 


17.5 


45.2 


4 


68 


29.8 


75.0 


5 


32 


14.0 


89.0 


6 


15 


6.6 


95.6 


7 


6 


2.6 


98.2 


8 


4 


1.8 


100.0 


TOTAL 


228 


100.0 





Test Conditions. Of the 228 students responding to Prompt 1, 116 
responded with oral teacher direction and prewriting; 112 responded to the 
prompt with oral teacher direction but no prewriting. A comp?v«.son of Tables 4 
and 5 suggests that there was no major difference between students who did 
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prewriting and those who did not. Slightly more unscorable responses were 
found under Condition 2 (8.9 percent), than for those writing under Condition 
1, (2.6 percent). 



TABLE 4 GRADE: 


3 PROMPT: 1 


COND: 1 


HOLISTIC SCORE 


VALUE 






CUM 


FREQ'IENCY 


PERCENT 


PERCENT 


0 


3 


2.6 


2.6 


2 


30 


25.9 


28.4 


3 


23 


19.8 


48.3 


4 


37 


31.9 


80.2 


5 


U 


9.5 


89.7 


6 


8 


6.9 


96.6 


7 


3 


2.6 


99.1 


8 


1 


.9 


100.0 




116 


100.0 





TABLE 5 GRADE: 3 


PROMPT: 1 


COND: 2 


HOLISTIC SCORE 








CUM 


VALUE 


FREQUENCY 


PERCENT 


PERCENT 


0 


10 


8.9 


8.9 


2 


20 


17.9 


26.8 


3 


17 


15.2 


42.0 


4 


31 


27.7 


69.6 


5 


21 


18.8 


88.4 


6 


7 


6.3 


94.6 


7 


3 


2.7 


97.3 


8 


3 


2.7 


100.0 




112 


100.0 
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Sample Response to Prompt 2: "How I Got My Favorite Possession" 



Score: 1 

I got my model three weeks ago it is green and has lots of other 
colers and the hachet opens and it has lots misels. when I got it 
it took four days to put it to gether. I was happy when I got it. It 
is importent to me becuse it belongs to me. 

Score; 2 

My bike is speshel to me because I can go lots of places on it. I 
don't have to use all my energie to go some were. And its all so fun 
to ride around. I all so like to my freinds house. It is all so fun 
to go off jumps on my bike. I go*, my bike for Christmas I was so happy 
when I got it. It was a year when it didn't snow so I went outside 
with my dad and rode it around. I have fun riding it around the 
naboorhood. Sometimes my seet fall> off when I come to school. But I 
don't care I just put it back on. T have fun riding it around. 

Score: 3 

One of my favorite possesion is a china doll. I got it when I turned 8 
it was a present from the Lion House. It looked like a pioneer it has 
a bonnet on and a long dress and it has pretty hair and small white 
socks it has black shoes and flowers on the drejs and bonnett. 
Whenever I think about it I take it off the shelf and I hug it. One 
time it almost broke, I took it and hugged it and then started to cry. 
By that experience I know it is my most valuable possesion. 

Score: 4 

On February 11, 1987 was my birthday. After I had dinner and when I 
was eating my cake I opened my presents. One of the presents was 
wrapted in nice wrapping paper. Under the box was a beautiful doll. 
She was a My Mother Pet kind. I wanted one so bad! See Diane my 
sister has one called Lisa. But now I have one called Cami ! She was 
so beautiful she looks just like Lisa but a little bit different. She 
has dark ebony hair and beautiful brown eyes and a light blue dress and 
bonnet. My mom gave her to me but I have to be careful with her 
because she's expensive. But i love her very much! 



Summary of Results for Prompt 2 

Prompt 2 asked students to identify a special possession and to both 
describe it and provide a narrative o? the occasion when they received the 
object. This task askeo for some complexity in writing because of the emphasis 
on both description and narrative; the purpose of the \:ask was to determine if 
the student could integrate the description into the narrative. No particular 
audience was specified, and no format other than the emphasis on story was 
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provided. Students, again, drew on personal ixperience to supply the necessary 
information. 

Frequencies and percentages uf scores. Two hundred thirty-five (235) 
students responded to Prompt 2, which was tested under Condition 1 and 2. The 
mean for the responses was 3.80, with a standard deviation of 1.60. About 55 
percent of the students scored a 4 or above on this writing task; 13 percent 
achieved a ranking of 6 or above, while 1.7 percent achieved the highest 
ranking of 8. Table 6 below provides summaries of the percentages and 
frequencies at each score point. 



TABLE 6 GRADE: 


3 PROMPT: 2 


HOLISTIC 5J0RE 


VALUE 






CUM 


FREQUEr':Y 


PERCENT 


PERCENT 


0 


6 


2.6 


2.6 


2 


48 


20.4 


23.0 


3 


SO 


21.3 


44.3 


4 


57 


24.3 


68.5 


5 


43 


18.3 


36.8 


6 


16 


6.8 


93.6 


7 


11 


4.7 


93.3 


8 


4 


1.7 


100.0 




235 


100.0 




Test conditions. 


Of the 235 


students ' 


.ending to Prompt 2, 112 



respor.ded with oral teacher direc-ion and prewriting; 123 responded with oral 
teacher direction but no prewriting. A comparison of Tables 7 and 8 suggests 
that there was no major difference between students who did prewriting and 
those who did not. The number of unscorable responses was identical. Tables 7 
and 8 provide a summary of the percentages of responses at each score level as 
wen as the frequency counts for each condition. 
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Q DDHIIDT • 

o rKUnrl I 


c LUNU : 1 


HOLISTIC SCORE 








CUM 


UAI IIP 




DCDrCKIT 

rtKLtN 1 


PERCENT 


0 


3 


2.7 


2.7 


2 


21 


18.8 


21.4 


3 


22 


19.6 


41.1 


4 


28 


25.0 


66.1 


S 


24 


21.4 


87.5 


6 


8 


7.1 


94.6 


7 


S 


4.5 


99.1 


8 


1 


.9 


100.0 


TOTAL 


112 


100.0 





TABLE 3 GRADE: 


3 PROMPT: 


2 COND: 2 


HOLISTIC SCORE 








CUM 


VALUE 


FREQUENCY 


PERCENT 


PERCENT 


0 


3 


2.4 


2.4 


2 


27 


22.0 


24.4 


3 


28 


22.8 


47.2 


4 


29 


23.6 


70.7 


5 


19 


15.4 


86.2 


6 


8 


6.5 


92.7 


7 


6 


4.9 


97.6 


8 


3 


2.4 


100.0 


TOTAL 


123 


100.0 
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Sample Responses to Prompt 3: An Unusual Day at School 

Score: 1 

I like school it is fun for me and boys and girl. 

All of the boy and girl have fun at school. 

I like readding at P-- school it is fun readding. 

I like readding it is fun to read at school. 

I like to go to school at 7:00 it is fun. 

I like to paly games at school it is fun for me. 

All of the big boy like to paly with me to. 

I like to paly with a boy and two boy to. 

I lik» to paly with my f rends it is fun. 

A lot of f rends cum to paly with me. 

Score: 2 

The first day of scool was very scairy their was diffent people and the 
room was very diffent. So we got uos to the class room. So now it was 
time to go home. So some people were very poshe so the firs day so that's 
what happen Somebody ran very fast a I fell on the grass. And rviom was 
watting for out in the car. and my pains had dust on eim so she said 
let's go home and wash them so we did. and then she said that was not 
dust it was dirt. So I said well can I go to bed now so that night my mom 
whent to the stor and got me a ice cream so she for got I was in bed. So 
she came and so she came down stairs and woek me up and I said go away 
I am triing to sleep, so I got up and ate the ice cream. 

Score: 3 

Mu anushul da of school was September 1, 1987. It was the first day of 
school. I was fritened of my new class because I had never been in therd 
grad befor. I am use to my calss now becuse I have been in therd grade 
long enuff that I know evrybody in my class. I'd like to tell you about 
the firt day of school. Well it all hapend like this. I was on my way to 
school with my firend Amanda. And when I got to school. It was ok untill 
I got to my class. It got real scary becuse I had a new techer. And 
their were alot of new chilren in my class. That's why I was fritened. 
Anyways I got my desk. And then she gave us suplys that we would nead. 
My farnt part whas when we got to know our techers name. He** name was 

Mrs. R. . She is pritty she has brown hair. And she smiles all the 

time. Evrytime the hole class is quit she is so nice, she even puts 
marbels in the jar. Then we played games and got to know evrybody in the 
class. And by the time school got out I thote it was the best thing that 
ever happend to me. While I was walking home thot about arhool that day. 
When I got home I told my mother about my first day of school. And I have 
loved school. I like Mrs. R most of all. 

Score: 4 

Let me tell you about my unusual day at school. O.K. thank you. My 
unsual day at school is in 2nd grade. I was eating lunch one day and ir.y 
tooth came out. I was suripesed that it came out, I almost swallowed it. 
When it came out and I put my tounge in the emptey spot it felt wierd. I 
took my tray and cleared it and went to my teacher. She tolled me to 
rince it out with water, and I did. And did I tell you I betted my mother 
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1 doller and I had to pay her. I thot it would come out after Stant 
Patricks day. But I was wrong. And she was right. And I did not show 
her tell ^e got home. She looked spriesed. And I whented to give her the 
doller and SHE WOULD NOT TAKE IT! She said I would have to work it off. 
AND I HATED TO DO WORK! THE END. 



Summary of Results for Promnt ^ 

Prompt 3 required students to identify an unusual day at school; students 
were encouraged to focus on specific events rather than chronicling an entire 
day. The objective of the task was to determine whether students could 
identify specific elements or events in a day and focus a narrative on thost. 
no specific audience was identified. Students obviously were forced to draw 
upon their personal experiences to satisfy the writing task. 

Frequencies and percentages of scores. Two hundred and two (202) students 
responded to Prompt 3, which was tested under Conditions 1 and 2. The mean for 
the responses was 3.62, with a standard deviation of 1.95. Approximately W 
percent of the students scored a 4 or above on this writing task; 14 percent 
achieved a ranking of 6 or above, while 4 percent achieved the highest ranking 
of 8. Table 9 provides summaries of the percentages and frequencies at each 
score point. 
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FABLE 9 GRADE: 3 PROMPT: 3 HOLISTIC SCORES 



VALUc 






CUM 


FREQUENCY 


PERCENT 


PERCENT 


0 


18 


8.9 


8.9 


2 


43 


21.3 


30.2 


3 


33 


16 3 


46.5 


4 


49 


24.3 


70.8 


5 


30 


It. 9 


85.6 


6 


10 


5.0 


90.6 


7 


11 


5.4 


96.0 


8 


8 


4.0 


100.0 


TOTALS 


202 


100.0 





Test Conditions. Of the 202 students responding to Prompt 3, 99 responded 
with oral teacher direction and prewrit.ing; 103 responded with oral teacher 
direction but no prewriting. A comparison of Tables 10 and 11 suggests that 
there were differences at almost every score point between students who did 
prewriting and those who did not. All score points except one for students who 
did not do prewriting show a higher frequency count. The number of unscorable 
responses under Condition 2 was only slightly higher than that for Condition i. 
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IMdLl lU uKMUl. 


o rKUHPT: 


3 COND: 


1 HOLISTIC SCORE 


VALUE 






CUM 


FRFOUFNfY 




DCDrCKIT 

rtKLtli 1 


0 


8 


8.1 


8.1 


2 


29 


29.3 


37.4 


3 


22 


22.2 


59.6 


4 


24 


24.2 


83.8 


S 


11 


11.1 


94.9 


6 


4 


4.0 


99.0 


7 


1 


1.0 


100.0 


TOTAL 


99 


100.0 





TABLE 11 GRADE: 


3 PROMPT: 


3 COND: 2 


HOLISTIC SCORE 


VALUE 






CUM 


FREQUENCY 


PERCENT 


PERCENT 


0 


10 


9.7 


9.7 


2 


12 


13.6 


23.3 


3 


11 


10.7 


34.0 


4 


25 


24.3 


58.3 


5 


19 


18.4 


76.7 


6 


6 


5.8 


82.5 


7 


10 


9.7 


92.2 


8 


8 


7.8 


100.0 


TOTAL 


103 


100.0 
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Sample Responses to Prompt 4: "A Scary Experience" 



Score: 1 

In a store I got lost. I thought I was going to get kidnaped. I looked 

all over the store. And then I found her. I was so scared I souldn't 
find her. I cryed and cryed. I was so glad I found her. 

Score: 2 

One day we wen't to the mall. I sor.'ted to look at the shoes and we did, 
then my mom, dad left I didn't no. 

I was still looking at the shoei then I stoped looking and I saw that my 
mom, dad wher gon I started to look for them for a long time untill I 
fowned them and I was happy. The End. 

Score: 3 

Once I was trick or treating and we wnet to 20 houses and one persen said 
that people were in a truck throwing punkins at people. Then I heard a 
couple of min. later someone crying someone hit him in the leg with a 
pumkion. Then the next day I found out my brothers friend got hit in the 
head his name is Mat P--. Him and Dany my brother was trick or treating 
when this happend. 

The reasen this is scary to me is I thout I woud get hit. The end. 
Score: 4 

One day af rer school when school was over and everybody was gone and this 
bully came up to me. He said to me. "Do you want to fight." I said 
"no." Then his friends came along. Then his friend said to me, "If you 
don't fight him I will bring my dog named "killer, boy was I scared. 
When they where talking I just run home. But they almost coult me. Boy 
was I liicky and I told my mom and dad. The next day bully got in trouble 
by the preisabol. I was happy that day. That day it was just fine. 



Summary of Results for Prompt 4 

Prompt 4 asked students to identity a scary experience; the emphasis was 
placed on describing the experience within a narrative framework and keeping a 
focus on a particular event rather than a series Students had to draw upon 
their personal experience for this prompt and were not given any format other 
than the narrative clue. No particular audience was specified. 

Frequencies and percentages of scores. One hundred ninety-eight (198) 
students responded to Prompt 4 which was tested under both Condition 1 and 2. 
The mean for the responses was 3.31, with a standard deviation of 1.76. About 
48 percent of the students scored a 4 or above on this writing task; 13 percent 
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achieved a ranking of 6 or above, while .5 percent achieved the highest ranking 
of 8. Table 12 below provides summaries of the percentages and frequencies at 
each score point. 



TABLE 12 GRADE: 


3 PROMPT: 


4 HOLISTIC SCORE 


VALUE 






CUM 


FREQUENCY 


PERCENT 


PERCENT 


0 


20 


10.1 


10.1 


2 


49 


24.7 


34.8 


3 


34 


17.2 


52.0 


4 


51 


25.8 


77.8 


5 


18 


9.1 


86.9 


6 


21 


10.6 


97.5 


7 


4 


2.0 


99.5 


8 


1 


.5 


100.0 


TOTAL 


199 


100.0 





Test Conditions. Of th9 109 students responding to Prompt 4, 99 responded 
with oral teacher direction and prewriting; 99 responded with oral teacher 
airection but no prewriting. A comparison of Tables 13 and 14 suggests that 
there were some differences at almost every score point between students who 
did prewriting and those who did not, but no overall consistent pattern 
appears. The number of unscorable responses u 'r Condition 2 was over 50 
percent higher than for Condition 1. 
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TABLE 13 GRADE: 


3 PROHPT: 


4 COND: 1 


HOLISTIC SCORE 


If A 1 ttC 

VALUE 






CUM 


FREQUENCY 


PERCENT 


PERCENT 


0 


6 


6.1 


6.1 


2 


26 


26.3 


32.3 


3 






45. 5 


4 


29 


29.3 


74.7 


5 


10 


10.1 


84.8 


6 


13 


13.1 


98.0 


7 


2 


2.0 


100.0 


TOTAL 


99 


100.0 





TABLE 14 GRADE: 3 PROMPT: 


4 COND: 2 


HOLISTIC SCORE 


VALUE 






CUM 


FREQUENCY 


PERCENT 


PERCENT 


0 


14 


14.1 


14.1 


2 


23 


23.2 


37.4 


3 


21 


21.2 


58.6 


4 


22 


22.2 


80.8 


5 


8 


8.1 


88.3 


6 


8 


8.1 


97.0 


7 


2 


2.0 


99.0 


8 


1 


:.o 


100.0 


TOTAL 


99 


100.0 
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Grade 8 Results 

Eight hundred and sixty-two (862) eighth graders provided the population 
for the eighth grade assessment. Unlike the third graders, the eighth graders 
wrote in four different modes: response to a request for information, 
narrative, persuasive speech, and informative report. Because students were 
responding in different forms and for different purposes, separate scoring 
rubrics had to be developed for each of the four prompts. 

In addition, the eighth graders wrote under different conditions than did 
the third graders; with all prompts but one, the eighth graders wrote under 
Condition 4--silent reading of prompt without teacher direction or prewriting. 
"rompt 6, which was identical to Prompt 2 for the third graders, was tested 
under all four conditions. However, in all cases, students had the same amount 
of time for writing--45 minutes--and the writing samples were first drafts. 
The total number of writings by prompt and condition along with the mean for 
each prompt appears in Table 15. 

On the following pages appear sample papers for each prompt used at the 
eighth grade level. Each paper represents a scoring category on the rubric for 
a particular prompt. (Readers may wish to refer to Section II of this report 
to review the eighth grade writing prompts.) The rubric for a particular 
prompt appears first, followed by the representative papers; they reflect a 
possible level of achievement at each score point. Within each score category, 
of course, there is always some variability. A brief summary of the results 
for the writing of each prompt appears after the sample papers. 
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TABLE 15: Total Number of 8th Grade Writings by Prompt and Condition 



Prompt Title 


1 


Condition 
2 


3 


Total 

4 




Oral w/ 
Prewrite 


Oral w/o 
Prewrite 


Silent w/ 
Prewrite 


Silent w/o 
Prewrite 


Letter (5) 








160 Count 160 
3.7J» Mean 3.78 
1.58 Std. 
Dev. 


Possession (6) 


114 Count 
4.27 Mean 
1.72 Std. 
Dev. 


70 Count 
3.44 Mean 
1.51 Std. 
Dev. 


79 Count 
4.92 Mean 
1.58 Std. 
Dev. 


150 Count 413 
4.07 Mean 4.18 
1.49 Std. 1.63 
Dev. 


Music (7) 








148 Count 148 
3.59 Mean 3.59 
1.67 Std. 1.67 
Dev. 


Snakes (8) 








141 Count 141 
4.39 Mean 4.39 
1.79 Std. 1.79 
Dev. 


Totals 


114 Count 
4.27 Mean 
1.72 Std. 


70 Count 
3.44 Mean 
1.51 Std. 
Dev. 


79 Count 
4.92 Mean 
1 58 Std. 
Dev. 


599 Count 862 
3.95 Mean 4.04 
1.65 Std. 1.68 
Dev. 
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SCORING RUBRIC FOR RESPCM)ING TO A REQUEST FOR INPOPMATION: PROMPT 5 



OOMEF i' 


4 

Supply all the 
requested infontaticxi 


3 

Supply all the 
requested inf ornatim 


2 

Supply most of the 
requested informatioi 


1 

Make minimal attenpt 
to si^ly the 
requested information 


DEVEIOI91E21T 


Use specific details 
and exairple^ to 
develop ideas 


Use some specific 
details a id exan^ les 
to develop ideas but 
not necessarily on a 
consistent basis 


Use few details or 
exanples to develop 
ideas; sane listing 
may appear 


Use few, if any, 
details or exanples; 
naming or listing 
will frequently 
appear 




Denraistrate a clear 

and logical sense of 

order throu^ 

appropriate 

parr iraphing and use 

of .ansitions 


Demonstrate a feli*ly 
consistent sense of 
order through 
paragrsqphing but may 
be inconsistent in 
use of transitions 


Reveal seme sense of 
order, but may be 
iTiCcnsistent in the 
use of paragr8q;)hs 
ancVor transitions 


Demonstrate 
inconsistency in 
ordering ideas and 
Inconsistent use of 
paragraphs and 
transitions 


SYMEAX 


UGnionsuate maturity 
and variety of 
sentence structure as 
well as vocabulary 
for audience and 
purpose 


Demonstrate variety 
and control of 
sentence stnx:ture 
and vocabulary 
appropriate for 
audience and purpose 


Demonstrate some 
variety and control 
of s^fibence srmcture 
but control of 
vocabulary may be 
irregular 


Demonstrate 
inraaturity in 
sentence stmcture 
and show limited 
range of vocabulary 


AUDIENCE 


ui utsJuouai u inuenQeci 

audiencp*s needs and 
address those nec^s 
clearly and with 
ccxisistent point of 
view 


unaerstana intenaea 
audience's needs and 
usually address those 
needs clearly and 
with a ccaisistent 
point of view 


i!?y or may not 
address the 
audience's needs and 
will reveal 
inconsistencies in 
point of view 


Show little attenti(^ 
to intended 
audience's needs and 
display no consistent 
point of view 
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PROMPT 5 



OONVEOTIONS 


Exhibit only mininal 
errors, if any, in 
usage and inechanics, 
none of vAiich detract 
from cxnprdiension of 
the text; will 
di^lay clearly 
legible handwriting 


E)diibit seme errors 
in usage and 
mechanics but these 
do not detract from 
ocxrprehension of the 
text. Handwriting is 
clearly legible 


Display fteqiient 
mechanics and usage 
errors ^ch detract 
front the text; 
handwriting will not 
always be legible 


Di^lay numerous 
mechanics and us€ige 
errors vAiidi detract 
seriously from the 
text; handwriting 
frequently will be 
illegible 


IfODE 


RespOTid in the 
specified format 


Respcxid in the 
specified format 

1 


Respcxid ir the 
specified format Ixit 
ignore some elements 


Shew little 
understanding of 
specified format 



SAMPLE PAPERS FOR RESPONSE TO PROMPT 5: LEHER TO KELLY* 



Score: 1 

Dear Kelly, 

Our school here is a good school. We have alot of fun activities and 
teams We have the Volley ball team, track, gymnastics, intermurals. Some 
activities, we have there are the dances that we have about 5 or 6 times a 
yecr Try to get your things together and meet new friends because once 
you do that you can conssintrait on your homework becuse good grades are 
v9ry important. Stay AWAY from Mr. R. because hes mean. We have a great 
track areas, our school yard ir very big. School starts at 8:15 A.M. they 
sometimes have lock outs and thats when you stay after school for a half 
hour. Here are some teachers that I can name to kind of help you out 
theres Mrs. D, Mrs. D., Mr. E., Mr. B., and Ms. P and that's all the 
teacher I know I hope you enjoy it here at 

Score: 2 

Dear Kelly, 

I hope you will like it here at our school. The place is very easy to 
get around. I'm sure you will have no trouble. There is only one level 
to it, I guess that makes it easy. The front of the school faces North. 
There is two ways to get in from the front 1st, the main door, the one 
that is so far back compared to the other school. The buses unload and 
load there. We have 5 buses. Now, the easiest way to find your way 
around, use the main door that leads you to the Main Office, and when you 
enroll, you'll get a map of the school, so you'll know where you're going. 

About the teachers, most of them are O.K., but some of them you won't 
want to get stuck with. We have two 8th grade English teachers, Mrs. E. 
and Mr. K, he's cool, I think you'll like him. We have to girls gym 
teachers. Miss B. and Mrs. L, and two boys Coach H. and Mr. M. 3 science 
teachers; Mrs. G., Coach D. (he used to teach gym) Mr. S. 1 German 
teacher. Miss A (take german, it's easy and A is laal nice.) A Spainish 
teacher, Mrs. H, who also teaches type. 

Some of the Activities are dances, usually Halloween, Valentines, Day, 
and a Spring Dance. Some of the clubs are the Arrow Club, which requires 
I think a 3.0 or higher. Honor Roll, you need 5 honors on your report 
card, and the Blue and Silver Club I think is citizenship. Most of the 
places we go will usually require you be in the Arrow Club. The 9th 
graders get to do most of the fun stuff. 

We have a fun run at the beginning of the year. We have presidential 
elections for each grade once a year, and were doing that right now. 



* Names and addresses have been deleted from the samples. 
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The classes you will be able to take are required, English, History, 
Reading, Computers, Science, P.£. Selected; Foreign Language, Type, 
Speech, Drama, Home Ec, Wood Shop, Car Shop, Band, Choir, Sewing, and 
Beginning Foods. 

Well that covers most of the Basic stuff. Good luck and Hope you'll have 
fun. 

Score: 3 

Dear Kelly, 

Hi, my name is . Our school is pretty nice. We have really "ood 

teachers. They're all nice. 

We have many classes you can take here--band, french, german, Spanish, 
crafts, homaking, and etc. There are so many fun classes that even the 
really hard classes don't bother you as much. But don't think we just 
have fun in this school. We learn so much, so fast. We take required 
classes like history and science. Even though most people don't like 
those kind of classes they aren't that bad here. I mean the teachers are 
so nice and they help you so much, the hard classes are a little bit 
easier. 

And you wanted to know about activities, ./ell we have many activities. 
We have assemblies, and dances, and if you get good citizenship in school 
you can go on a ski trip and to lagoon. In our assemblies, we do many 
things, we have concerts, talent shows, drama, plays, and sometimes we 
just have them to talk about rchool. Once we even had a snake man come. 
Even though our assemblies are fun, they're still all learning 
experiences. Every year we have about 4 dances. They're raally fun toe. 
Of course, you can't get bad citizenship or you won't get to go on a ski 
trip. That's super fun. Even if you don't know how to ski, you can go 
and take lessons. About the end of the year we go to Ir.goon. Now that is 
one of the funnesl activities We leave for Bountiful as soon as school 
starts. When we get there we pay and we c?n do anything we want. 

If you ask me, our school is really fun. I like it a lot. What amazes me 
IS you learn so m'^ch and it's still so fun. well, I can't wait to meet 
you. 

Score: 4 

Dear Kelly, 

I'm answering your letter about the school I attend, the things that we do 
here, and the way we do them. 

Our scnool is a medium to large size for a junior high school. 

We have good teachers who really care about the kind of grades that we 
get. Most of them will do anything that they can to help you out. 
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Our principal is a pretty straight forward guy, he is nice. Sometimes 
alot of people think he's too strict on such things as the dress code, but 
those people are just basically trying to cause trouble for theirselves. 

At our school we have a lot of extra-cirricular classes available. We 
have band and orchestr;,, Spanish, German, and French, Karate, 
computer/keyboarding, type, and theatre arts or drama. 

We have quite a few assemblies, dances, and activities. We usually have 
an assembly once or twice a month, and they are usually very interesting, 
funny, or exciting. 

We have about three dances a year. Usually one in October, one in 
December, and on in April. In other words we have a Halloween Dance, a 
Christmas Dance, ind an Easter Dance. 

The activities that we have are usually really fun. 

I don't know if your school is on the '.rimester schedule or net, so I'll 
just tell you about it anyway. We have five classes a day. Each one 
lasts seventy minutes. Our school starts at 7:50 and gets out at 2:20. 
Sometimes I think that it should start at 8:00 and get out at 2:30. To me 
that sounds a little less strange. But, oh well, they must have a good 
reason for it. Our first class goes from 7:50-9:00. Second hrur goes 
from 9:05-10:15. After that half of the school has lunch until about 
10:40, while the rest of us go to third hour from 10:20-11:30. Then we go 
to lunch. Fourth hour is fiom 12:00-1:10, and fifth hour is from 1:15- 
2:20. Fifth hour has five minutes less than the rest of the classes. 

If you aren't used to it, it might sound a little confusing, but it's 
really not. 

Well, I hope you enjoy our school as much as I do. I'm sure you will meet 
alot of new people and make a lot of friends. I'm sure I'll see you 
around . 



53 



Summary of Results for Prompt 5 

Prompt 5 provid3d an opportunity for students to demonstrate two basic 
principles of writing: providing specific information in response to a request, 
and addressing a specific audience. The writing task was designed to provide 
students with a realistic situation--a potential classmate wanting to know as 
much as possible about the school where he or she might be enrol ling--and a 
familiar mode or format by which the student writer could respond—the letter. 
In addition, the prompt contains specific clues for what should be included in 
the appropriate response as Kelly asks for information on teachers, classes, 
and activities; the student writer also is given the latitude to provide 
additional information if so inclined. 

Frequencies and percentages of scores. One hundred sixty (160) students 
responded to Prompt 5. This prompt was tested only under Condition 4--silent 
reading without prewriting. The mean for the responses was 3.70, with a 
standard deviation of 1.47. Approximately 55 percent of the students scored a 
4 or above on this writing task; 17 percent achieved a ranking of 6 or above, 
while 1.3 percent achieved the highest ranking of 8. See Table 16 for 
summaries of the percentages and frequencies at each score point. 
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TABLE 16 GRADE: 


8 PROMPT: 


5 CONO: 4 


HOLISTIC SCORE 


VALUE 






CUM 




rtKLtn 1 


PERCENT 


0 


1 


. 0 


.6 


2 






28. 1 




26 


16.3 


44.4 


4 


44 


i. .5 


71.9 


5 


18 


11 3 


83.1 


6 


18 


11.3 


Q4 4 


7 


7 


4.4 


98.8 


8 


2 


1.3 


100.0 


TOTAL 


160 


100.0 
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SCXDRING RUBRIC FOR NARRATIVE EMPHASIZING PERSOIAL EXPERIENCE: FROMPT 6 



OOMTEXT 


4 

Provides rich 
descriptive back- 
ground of objects, 
places or people 
for esqperience 


3 

Provides descriptive 
background of 
objects, places, or 
people 


2 

Name objects, places 
or people with little 
elaboration; or over^ 
en|3hasizes context, 
drawing attention 
away from incident 


1 

Lists or names 
objects with no 
description 


mcimrr 


Narrates vivid 
account of personal 
involverjent in an 
expai&nce (**shows 
story*') 


Narrates personad 
involvement in 
e>q>»rienoe ("tells 
story") 


Lists action with 
soma narration of 
personal eaqperienx 


Has little or no 
dsvelopnBnt of action 
or personed ex- 
perience 


significance; 


Strongly aifiiasizes 
vAiy experienoa was 
in{x>rtant 


IDiplies or states Jtitiy 
experience war. 

ijll|)UI't£Ult 


Gives little indica- 
tion why eoqperience 
was i]i{)Qrtant 


Gives little or no 
indication of why 
ea^erienoe was 
inportant 


OOHEREMCE 


Displays veil- 
developed narrative 
stnx±ure (beginning, 
middle, end) ; or has 
theanatic unity 


Di^lays narrative 
structure (beginning, 
middle, end) ; or has 
thematic unity 


Displi^ limited 
narrative structure 
(particularly with 
middle or end) ; or 
has little thematic 
unity 


Di£pl«^ little or no 
develocment of 
narrative structure 
(beginning, middle, 
end) ; or lacks 
thanatic unil^ 


CONVENnONS 

Ri 


Has strcxig OMitrol 
over mechanics 


May have errors; 
these do not inter- 
fere with ccnmunica- 

tiCMl 


Has errors that may 
interfere with 
ocmmmicatiOT 


Has errors that 
interfere with 
oommunicaticxi 
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SAMPLE RESPONSES TO PROMPT 6: POSSESSION 



Score: 1 

My favorite possession \i my stuffed bunny, it's five feet tall it's a 
light, light yellow, it has long bushy ears with pink and blue in the 
middle. It has pink fee; and hands, it's eyes are blueish black, and it 
has a pink nose. It is wearing a shiny blue tuxedo with a matching big, 
top hat and a tail gaurd which covers his big, fluffy, bushy tail, the 
reason it is my favorite .jossession is because my Grandpa died right after 
he gave it to me and that is the only thing that reminds me of my grandpa 
because he died when I really little and I didn't know him to well. My 
favorite possession, which is my bunny, sits on my bed during the day and 
I lay him on the floor at night. Some people think I'm stupid to have a 
stuffed bunny for a favorite possession, they say it's like having a teddy 
bear. But I don't listen I just ignor them or tell them the reason. If I 
tell them the reason the^ usually understand. When I got my stuffed bunny 
it IS 2 1/2 feet tal'.ar than me. 

Score: 2 

My favorite possesion is a toy elephant that is sometimes cute and cuddley 
and other times is dirty and smelly. When he is cute and clean I play 
with him. He always keeps me company and I talk to him when I am feeling 
down. When I woke up Christmas morning I had no idea it was Christmas 
becuse I was only six months old. My mom brought me out to the front room 
and the first thing I noticed the tree with the blinking lights. Then my 
mom gave me the doll. I just looked at it. 

This doll that named baby bo bo, played with me all my life. I would play 
house and he would be my baby. He sits up. He looks like a regular doll 
with big ears and a big nose. I played with him so much that his button 
eyes fell off and my mom took a magic marker and drew dots for eyes. He 
is as old as me, fourteen years. He still is my favorite friend. He 
always talks to me. 

When I am sick or don't feel like doing anything he comes out from his 
hiding spot and talks to me. Also he sleeps with me if I want him to. 

Sometimes when I used to play with him. Baby Bo Bo would fall in some dirt 
and get dirty. Or he would get some food on him because he was a sloppy 
eater. If this happened my mom would wash him in the washer and then she 
would dry him in the dryer. After he was washed he felt warm and fuzzy. 

Now that I have explained my favorite possession, I hope you will get an 
elpehant too. 

Score: 3 

One Christmas morning about 3 years ago, I went into the living room to 
see what santa had brought me. There were many wonderful gifts for me, 
but the one I noticed the most was a beautiful Keepsake doll. 
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It was not an ordinary doll because my mon had made it. The doll was very 
soft and her skin had a plush color to it. The doll was wearing a navy 
blue dress with real light orange flowers, it had white bloomers under the 
dress, and a white apron over it. She had brown hair made out of yarn and 
curled into ringletts with straws. Her eyes were blue to match her dress, 
and her lips and cheeks were a rosy pink. She had black felt shoes with a 
clear buttom to look like a holder. She had a silky white ribbon that 
pulled back part of her h?Ar. She was very beautiful. She was on a 
special doll holder stand so I could keep her on my dresser. The first 
time I evf., saw it, I had the feeling of aw, and I knew my mom loved me 
because she wouldn't have taken the time and patience to make that 
beautiful, neat, and special doll, if she didn't love me. So that's why 
my keep sake doll, is so special to me. 

Score: 4 

It was a cold, windy, autumn day. My brothers and I were upstairs playing 
like we knew karate. We were in our pajamas, kicking and punching. But 
we wouldn't really even touch the other person. All you could hear was 
yelling and jumping from downstairs, which ment that we were having fun. 

One of my brothers went downstairs. But that didn't stop us from playing. 
We could fight without him. Then two more went downstairs. I just dept 
on playing. I thought that they might just be going down to get a drink 
or something. Then I heard my mom and dad's voices. They had gone out 
for the night, but I guess they came back. I suddenly heard the voices 
get louder. They were coming upstairs. My brothers all ran upstairs 
trying to beat my parents. They had big grins on their faces. I wondered 
if mom had brought ice cream home, or malts. But my brothers would have 
theirs by now if they did. Then I looked at my dad. He had a golden 
little ball of fur in his arms. My eyes wattered. I knew what it was 
because I had been asking for one for a long time. Then a long snout 
looked up at me and struggled to way its tail in my dads arms. When he 
got to the top of the stairway, he let it down. It walked towards me with 
its little tail wagging 50 miles an hour. I picked it up with a big 
smile. It licked me all over my face. I loved it all ready and I had a 
feeling that it loved me too. I looked at my mom and dad and said "Thank 
you" at least 7 times. I looked at the puppy again and asked what it was, 
a girl or Boy. It was a girl. So I named her Cindy. She had a long cute 
snout with a black nose. And she had little golden ears. And huge paws! 
I loved Cindy a whole bunch. You see, I had wanted a dog for over a year 
now but it had to be a golden retriever because chose are my favorite 
kinds of dogs. And I sure got one! Cindi , my puppy! 
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Sumtnarv of Results for Prompt 6 

Prompt 6 was the same writing tasK as that performed by third graders, 
except that at the eighth grade level this task was tested under all four 
conditions. The prompt asked students to combine narrative and description, in 
this case about a favorite possession. Pirections suggest that the writing 
should focus on the time the writer received the possession. In addition, the 
prompt asked students to tell how they felt about the possession and why the 
possession was important to them. No particular audience was specified. 

Frequencies and percentages of scores. Four hundred and thirteen (413) 
students responded to Prompt 6. The overall mean was 3.88, with a standard 
deviation of 1.50. Approximately 65 percent of the students scored a 4 or 
above on this writing task; 21 percent achieved a ranking of 6 or above, while 
3.4 percent achieved the highest ranking of 8. See Table 17 for summaries of 
the percentages and frequencies in the rankings. 



TABLE 17 GRADE: 


8 PROMPT: 


6 HOLISTIC SCORE 








CUM 


VALUE 


FREQUENCY 


PERCENT 


PERCENT 


0 


3 


.7 


.7 


2 


74 


17.9 


18.6 


3 


65 


15.7 


34.4 


4 


108 


26.2 


60.5 


5 


74 


17.9 


78.5 


6 


54 


13.1 


91.5 


7 


21 


5.1 


96.6 


8 


14 


3.4 


lOC.O 


TOTAL 


413 


100.0 
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Test conditions. Of the 413 students responding to this prompt, ii4 
responded with oral teacher direction and prewriting; 70 responded to the 
prompt with oral teacher direction but no prewriting; 79 responded to "he 
prompt with silent reading but with prewriting; ind 150 responded with silent 
reading and no prewriting. 

A comparison of Tables 18-21 suggests some differences at a number of 
score points between the students writing under Condition 1 and those writing 
under the other conditions. Most notable, perhaps, is that the highest 
percentage of students achieving an 8 score (7 percent) appeared under 
Condition 1 while the lowest appeared under Condition 4 (.7 percent). Students 
writing under Condition 1 did markedly better at all levels than students 
writing under Condition 2, but students writing undjr Condition 3 showed a 
marked increase in the percentage achieving in the top three score categories 
(43 percent) as compared to those under Condition 1 (21 percent). The number 
of unscorable responses under condition 1 was identical to those under 
condition 3. 



TABLE 18 GRADE: 8 PROMPT: 


6 COND:^ 1 


HOLISTIC SCORE 








CUM 


VALUE 


FREQUENCY 


PERCENT 


PERCENT 


2 


20 


17.5 


17.5 


3 


20 


17.5 


35.1 


4 


29 


25.4 


60.5 


5 


20 


17.5 


78.1 


6 


12 


10.5 


88.6 


7 


5 


4.4 


93.0 


8 


8 


7.0 


93.0 


TOTAL 


114 


100.0 
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TABLE 19 GRADE: 8 PROMPT: 6 COMD: 2 HOLISTIC SCORE 



VALU£ 


FREQUENCY 


PERCENT 


PERC 


0 


2 


2.9 


2.9 


2 


20 


28.6 


31.4 


3 


16 


22.9 


54.3 


4 


17 


24.3 


78.6 


5 


7 


10.0 


88.6 


6 


7 


10.0 


98.6 


8 


1 


1.4 


98.6 


TJTALS 


70 


100.0 





TABLE 20 GRADE: 8 PROMPT: 6 



COND: 



HOLTSTIC SCORE 



VALUE 



2 
3 
4 
5 
6 
7 
8 



TOTAL 



FREQUENCY 

7 
7 

19 
12 
24 
6 
4 

79 



PERCENT 

8.9 

8.9 
24.1 
15.2 
30.4 

7.6 

5.1 

looTo 



CUM 

PERCENT 

8.9 
17.7 
41.8 
57.0 
87.3 
94.9 
100.0 
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TABLE 21 GRADE: 


8 PROMPT: 


6 CONO: 


4 HOLISTIC SCORE 


VAI UP 






CUM 


rncnilCMr'V 


PERCENT 


PERCENT 


1 ■ 


1 


.7 


.7 


0 

c 


07 
CI 


18.0 


18.7 


3 


22 


14.7 


33.3 


4 


43 


28.7 


62.0 


5 


3b 


23.3 


85.3 


6 


11 


7 3 




7 


10 


6.7 


99.3 


8 


1 


.7 


100.0 


TOTAL 


150 


lOC.O 
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SCORING RUBRIC FOR CCMiKySWG A PERSUASIVE SPEECH: FROMFT 7 



OMfTENT 


4 

Address the issue 
cleeurly and establish 
and maintain a 
cxHisistent positicxi 
CXI the issue 


3 

Address the issue 
clearly and establish 
and maintain a 
ccvisiscent position 
on the issue 


2 

Reflect seme inccm- 
sistency in identic- 
ing the issue and 
develq3ing a position 


1 1 

Exhibit difficulty 
establishing a 
position or defining 
the issue 


DEVELOFMENT 


Support position with 
specific exanplui, 
facts anVor personed 
experience clearly 
related to issue 


Support po6itic»i with 
exanples, facts 
ancV<^ perscml 
experience but with 
sanswhat less detail 
than a **4** pap&c 


Provide only general 
exanples or facts to 
scpport a position; 
may reflect some 
contradictions in 
developing argument 


Use hio^y general 
exanples and facts, 
some of whidi may not 
be relevant to the 
issue 


ORGANIZATION 


Demonstrate a clear 
and logical sense of 
order throuc(h 
appropriate para- 
graphing and 'jse of 
uransxuiofis 


Demonstrate a fairly 
consistent sense of 
order through 
paragraphing but may 
be inconsistent in 
use or transiuons 


Reveal seme sense of 
order, but may be in 
consistent in the use 
of paragraphs and^or 
transitions 


Demonstrate inconsis- 
tency In ordeririLi 
ideas and inconsis- 
tent iise of para- 
grcftis and transi- 
tions 


SYNTAX 


nenionstrate maturity 
and variety of 
sentence structure as 
well as vocabulary 
appropriate for 
audience and , ^' pose 


Demonstrate variety 
and control of 
sentence structure 
and vocabulaxy 
appropriate for 
audience and purpose 


Demonstrate seme 
variety and control 
of sentence structure 
bi't control of 
vocabulary may be 
irregular 


Demonstrate im- 
maturity in sentence 
structure and show 
limited range of 
vocabulary 


AUDTENrp 

7^ 


uTiciersuanu uPp 
intended atdienc3*s 
needs and addrcs3 
those needs clearly 
and with a consist^ 
point of view vihile 
also acknowledging 
opposing views 


Adaress the aual*-* 
ence*8 needs vAiile 
showing a' ^reness of 
opposinr tews 


May or may not 
address the intended 
audience's . >eds and 
will show little 
awareness of opposing 
views 


Shew little attention 
to intended audien- 
ce's needs and 
dli^lay no consistent 
point of view or 
awareness of opposing 
views 

if 



PROMPT ? 



2 



ODNVENTIONS 


E)diibit only minimal 
errors, if any, in 
usage and mechanics, 
none oj. wnxcn oeuBcc 
from conprehensicxi of 
the text; will 
display clparly 
legible handwriting 


E>diibit some errors 
in usage and mech- 
anics but these do 
nou aecracc rrciri 
ccrpr^iension of the 
text. Handwriting is 
clearly legible. 


Display frequent 
mechanics and usage 
errors vihich detract 
•rnjin me uexu# 
handwriting will not 
always be legible 


Display numerous 
mechanics and usage 
errors v^ich detract 
serxoj!Sxy rrom me 
text; handwriting 
f reqL ntly will be 
illegible 


MODE 


itespond in the 
specified format 


Respond in the 
specified format 


May reveed uncer- 
tainty about ap- 
propriate format 


Show little under^ 
standing of ^)eclf ied 
format 



7- 
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SAMPLE RESPONSES TO PROMPT 7: MUSIC 



Score: 1 

I think rock music Ig a b.\d influence on some kids, because the people I 
know about that are hard rock music lovers. Are the people I dont like to 
hang around with, because there language. There attides about life. And 
the drugs that are mostly involed with the kids. 

I personitlily dont like hard rock music because it's not enjoy able to 
listen to and I mostly done understand what they are saying. Mosc of the 
people that do listien to hard rock music are in little groups and they 
dont have a great deal of friends. So people dont like to hang around 
them is because they are consitly in trouble for doing something they 
werent so pose to do. 

If I had a choose I would have all the hard rock music gone. 
Score: 2 

I think that in a way parents are right. But I also understand tne 
teenagers. Some music has a meaning but some is just like somebody 
screaming into a microphone put it on tape and aet paid lots of money for 
it. I don't have any simpathy for the teenagers who lik2 that music. 
Rock music is divided into parts. For i stances there is Soft Rock, Rock 
and Roll, Hard Rock, and Heavy Metal. Heavy Metal and some of Hard Rock I 
think we could do without. But kids do have rights if the parent dose'not 
want their child to list to what ever music they think is inappropreate 
they should take charge and do something but some parents don't care and 
it wouldn't be fair to take it all away. You've got to admit you must 
have liked rock and roll whe you were young. 

Thank you 

Score: 3 

Parents and students, 

I am here to talk to you about an issue that involves us all. The issue 
of rock music at school dances, and on the radio. 

I don't think all rock music is bad, but there is some the world could do 
without. This is what we should get off the radio and our. of the dances. 

Groups like the Jets and Lionel Richie aren't that bad. If you have never 
listened to them you really should, and then maybe you would agree with 
mo. 

The groups . am talking about that the world could really do without are 
the heavy metal group like Motley Crue and Def Leopard. All they sing 
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about is violence and such, they never sing about good holesome things. I 
know some of you students, and maybe even some of tne parents might really 
disagree with me on this but it's what I believe. 

When I was in the seventh grade, my mixed glee teacher asked the students, 
if they had any of those heavy metal group records, to bring them so we 
could play them backwards to see if there were any backwards messages on 
them. You know what? Every single of the songs we listened too 'had a 
backward message on it. One of the songs said, "It's fun to smoke 
mariguano." T really don't believe that people need that kind of thing. 

I ..jpe that today I have convinced you that we don't need heavy metal 
groups, and that all rock music isn't bad. 

Thank you for listening. 

Score : 4 

Today we are all gathered here to talk about rock music. Some of you 
parents want it banned from th« stations and from the school dances. 
Otriers of you don't think of it as a probleir. 

I agree a little both of the groups. Some rock music should be banned 
from radio stations and from dances, but then there is some good rock 
music too. And I'm sure many students will agree with me. 

For instance, heavy metal rock is something that can affect students in 
the wrong way. It is very loud and the lyrics aren't too great either. 
Some heavy metal if played backwards suggests wrong things. For example, 
drugs. 

Last year I had the opportunity to attend a youth meeting about music. 
They played a song forward and then backward. Going backwards it said, 
"Its fun to smoke marajuana." I haven't ever listened to that song again. 
I never realized that those groups of singers sneaked things like that in. 

So in a way I agree with those parents and students who want rock music 
banned. ' 

But new for the other group of parents and students who don't think of it 
as a r-oblem. This can be true If we let it. We will never know if rock 
music is good or bad if we don't listen to it. I have my own stereo and 
small clock radio. My sister has an advanced gheto blaster too. My mom 
and dad know that we know what kind of music is good or bad. I don't 
think they would have given us our radios If they had had any doubts. 

I enjoy rock music alot. I also enjoy other kinds of music, but I know 
that many students only like rock. If their parents are worried about the 
effects of the music, they should listen to the music too. They will 
find, like my parents did, that all rock music isn't that bad. If a bad 
song comes on, just turn it to another station. You can just about always 
find a good song on with all of the many rock stations. 

66 



7/ 



A.> for our rock station and our school dances, parents really shouldn't 
worry. The songs at the dances are always good songs, with no lyrics that 
can affect students in the wrong way. The station, though, we don't have 
alot of controll over. But I'm sure they don't play alot of heavy metal. 

We as students would prefer it if the parents would agree to le* rock 
music into the dances. 

This is how I fell as a student and I thank you for all of your 
cooperation and support. 



Summary of Results for Prompt 7 

Prompt 7 required students to anticipate a situation in which a written 
text would serve as the basis for an oral presentation. Effort? were made to 
keep the situation as realistic as possible. The selection of rock music as a 
controversial issue and an audience of both parents and students was intended 
to provide a reasonable access point for all students. They were asked to take 
a position on the issue and to adopt a persuasive approach, using their own 
experience as well as any other evidence they thought appropriate. This prompt 
was tested only under Condition 4--silent reading without prewriting. 

Frequencies and percentages of scores. One hundred forty-eight (148) 
students responded to Prompt 7. The mean was 3.52, with a standard deviation 
of 1.52. Approximately /3 percent achieved a ranking of 6 or above, while 1.4 
percent achieved the highest ranking of 8. See Table 22 for summaries of the 
percentages and frequencies in the rankings. 



TABLE 22 GRADE: 


8 PROMPT: 


7 COND: 


4 HOLISTIC SCORE 


VALUE 






CUM 


FREQUENCY 


PERCENT 


PERCENT 


0 


5 


3.4 


3.4 


2 


43 


29.1 


32.4 


3 


20 


13.5 


45.9 


4 


46 


31.1 


77.0 


5 


15 


10.1 


87.2 


6 


8 


5.4 


92.6 


7 


9 


6.1 


98.6 


8 


2 


1.4 


100.0 


TOTAL 


148 


100.0 
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SCX)RING 



RUBRIC FDR AN INFORMATIVE REPORT: PRCWPT 8 



commv 


4 

' Select relevant 
information and 
provide a focus 
through synthesis of 
the information; make 
generalizations and 
ocvinections frxan the 
facts given 


.3 

Select relevant 
informatim and 
provide a focus for 
it through synthesis; 
show some atten(>t at 
generalizing and 
making oonnectiois 
f rem the facts given 


2 

Provide a listing of 
tacts with some 
evidence of an 
attenpt to classify; 
little synthesis is 
evid^ although seme 
attaii)t may be made 
to ccnbine facts or 
rewrite them 


1 ■ ■ 

List facts; parrot 
informati(xi given; no 
attenpt to classify 
or to synthesize 


DEVEIDFMENT 


Use specific details 
and exanples to 
develop ideas 


Use seme Eqpecif ic 
details and exanples 
to develOD ideas but 
not necessarily on a 
consistent basis 


Use few details or 
exanples to develop 
ideas; some listing 
may appear 


Use few, if any# 
details or exanples; 
naming or listing 
will l^nequently 
appear; no generalize 
jjig or connecting 
will occur 


(M3ANIZATI0N 


Danonstrate a cleeu: 
and logical sense of 
order through 
appropriate para- 
graphing ejid use of 
transitions 


u5nurstrate a fairly 
consistent sense of 
order through 
paragraphing but nay 
be inconsistent in 
use of transitions 


Revieal some s^ise of 
order, but may be 
inconsistent in the 
use of paragrsqphs 
antVor transitiais 


Demonstrate iriaonsis- 
taicy in ordering 
ideas and Inoonsis- 
tent use of 
paragra^te and 
transitions 


SYNTAX 


DaiKxistrate maturity 
and V£uriety of 
sentence structure as 
well as vocabulary 
appropriate for 
audience purpose 


Demonstrate variety 
and control of 
sentence structure 
and vocabulary 
appropriate for 
audience and purpose 


Demonstrate some 
variety and control 
of sentence structure 
but control of 
vocabulary may be 
Irr^jgular 


Demonstrate im* 
maturity in sentence 
stnicture and show 
limited range of 
vocabulary 


1 AUDIENCE 


Understand the 
intended audience's 
needs aiYl ac^ness 
those needs clearly 
and with a consistent 
point of view 


Understand the 
intended audience's 

llctSJa cuU USUclXXy 

address those needs 
clearly and with a 
consistent point of 


May or may nnt 
address the intended 
auuxence s neeus ana 
will reveal inconsis- 
tencies in point of 
view 


Shew little atten^^on 
to intended audi- 
ence s neeos ana 
display no consistent 
point of view 
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PROMPT 8 



COtVmfTIiMS 


Exhibit only minimal 
errors, if any, in 
usage and mechanics, 
none of \rfiic4i detract 
from oowpre*"ensic»i of 
the text; will 
display clearly 
lerrible handwriting 


Exhibit some errors 
in usage and mechan- 
ics but these do not 
detract from ccp*— 
prehension of the 
text. Handwriting is 
clearly legible 


Display frequent 
mechanics and usage 
errors which detracL 
ft riH the text? 

handwriting will not 
always be legible 


Display numerous 
mechdnics and usage 
errors which detract 
seriouslv from the 
text; handwriting 
frequently will be 
illegible 


MODE 


RespcMid in the 
specified forroat, 
showing use of the 5 
W*s 


Respond in the 
specified format, 
showing OOTtrol of 
the 5 W»s 


Respond in the 
specified format but 
ignore sajB elements 
of thfj 5 W>s 


Show little under^ 
standing of specified 
format, ignoring most 
if not all of the 5 
W's 
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SAMPLE RESPONSE TO PROMPT 8: SNAKES 



Score: 1 

King Cobra 

His name is tinker bell, He is 3 ft, he is from india, he strikes very 
slowly, when scared, angered or nervous, h" raises his hood has a very 
deadly bite. This snake dosent move to snake charmers music. King cobras 
kill about 25,000 Indians per year. Large King cobras have enough enom 
to kill hundreds of people. 

python 

This 8 foot Guy is r.amed Crackin His home town is India it took 14 members 
of the audience to hold him. He is'nt like the Diamond Rattlesnake be 
cause a Python lays e^^s. He looks really lazy. He likes being by water, 
this snake is'nt poisonous but I wold^nd like to be bitten by him they 
have large appatites because they eat goats, pigs, and deer. 

Score: 2 

A man by the name John Burton, an expert on snakes came to an assembly at 

the Junior High and showed some snakes. He had 3 snake which were 

naied Elvis, Crackin and Tinkertsll. 

Elvis is an rattel snake which has long hollw f^.ngs and is most dangerous. 
Elvis, upper third body raises in to an s when ready to atack. He stabs 
his preay and is very posinus Elvis gives life to live babies. 

Crackin, is a very big snake he's a python over 8 ft long. John pickes 14 
people to come down and hold him. He lays eggs and very suggish. he 
isfound near water and is fonded not poisonous he kill by constricting i 
don't like the sound of that. He fonded to kill anbd eat goats, pigs and 
deer. One pythion in Malaysia ate a 14 year old boy so they are very 
dangerous. 

tinkerbell is a King Cobra. He kills over 25,000 people a year one corbra 
has enobventin to kill hundons of people He is 3 ft long and has a hood 
and strikes slowly their is no untidot to cure a person. He fites and 
chews his food and lives in india. 

this snake guy mus have nerves of steel. I hate snakes they are scarey. 
I'm glad that soiii(» people arc showing us kiiis more about exciting things 
like snakes. 



Score; 



lohn Burton, an exptrt on snakes, showed our school an assembly about 
them. He brought three snakes: a diamond rattle snake named Elvis, a 
king cobra named Tinkerbell, and a Python named Crackin. 
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Elvis, the rattle snake, has long hollow fangs and is the most dangerous 
rattler in the United States. Elvis also has hollow teeth like hypodermic 
needles. When Elvis rests he has his body curled up. When Elvis get*: mad 
the upper third part of his body raises into an S-shape, and strikes his 
prey with tne whole length of the S-shape. Elvis stabs his pray with his 
fangs and releases a venom. When Elvis eats, he swallows his food whole. 
Elvis does not lay eggs, instead he gives live babies. 

■"inkerbell, the king cobra, is a three foot snake from India. She attacks 
without being angered, but strikes slowly. She is a hooded cobra and 
bites and chews with short fangs as the venom runs into the wound, and has 
a deadly bite with no antibiotics for the venom. Tinkerbell does not move 
to snake charmers music but instead she sways to snake charmers movements. 
The king cobra kills 25,000 Indian natives a year and has encgh venom to 
kill hundreds of peonle. 

Crackin, the python, is an eight foot python from india. At the asembly it 
took fourteen people to hold Crackin up. Crackin and other pythons do lay 
eggs and are sluggish or lazy snakes that are usually found by water. 
Pythons are not poisonous, they kill prey by constricting. Large pythons 
kill gr *s, pigs, and deer. A python in Malaysia ate a fourteen year old 
boy. 

The assembly was neat and if you ever qet the chance to see it I would 
advise to do so. 

Score: 4 

We recently had an expert on snakes visit our school. His name is John 
Burton. He brought with him a variety of snakes, but the most impressive 
snakes by far were the Diamond Rattlesnake, King Cobra, and the Python. I 
could tell they were impressive by the ooo's and aahh's of the students. 
Mr. Burton had sort of a comical attitude toward the snakes because their 
names were Elvis, tinkerbell, and Crackin. 

Elvis, the diamond rattlesnake, is the most dangerous rattlesnake in the 
United States. His teeth are hollow so the poision may be injected 
easily. When resting, he lays with his body coiled, but when attacking 
prey, he raises the upper one-third of his body and rattles its rattle. 
After injecting poision he then swallows his food whole. It's interesting 
that the poision that kiTs the prey has no effect on the snake. Unlike 
the python, the diamond ,'attlesnake gives birth to live babies. 



Tinkerbell, a King Cobra, i^ only *hree feet long. It attacks whenever it 
is angered and sometimes when t's not. When attacking, the cjbra reveals 
its hood. It strikes slowly and bites and chews as venom runs into the 
wound. Sometimes we see on T.V. snake charmers with thier "magical" 
music, but this one doesn't move when "magical" music is played but it 
sways to charmers' movenents. It is a very deadly snake because it kills 
twenty-five thousand Indian people per year. Having an incurable venonm, 
a lare cobras venom can kills hundreds of people. 
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Crackin, the python, is eight feet long and it took fourteen students to 
hold it. The python is a sluggish, lazy snake and lays eggs. Unlike the 
otiier two it kills its prey b constricting its powerful muscles until the 
victim cannot breathe. Large pythons can kill goats, pigs, -'ser and a 
python in Malaysia ate a fourteen year old boy! 

I'm sure you enjoyed the assembly as much as I did and maybe Mr. Burton 
will be willing to return next year. 
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Summary of Results for Prompt 8 

Prompt 8 offered students the opportunity to demonstrate their familiarity 
with a particular format--the ^ 3ws story--as well as an opportunity to show 
their ability to synthesize and organize a collection of controlled data. 
Unlike other writing prompts where students were free to draw upon their own 
personal experience, all students wrote from the same basic data. An implied 
audience of other students was provided, and some clues on the opening of the 
story also offered mir*^ guidance than with other prompts. This prompt was 
tested only under Condition 4--silent reading with no prewritin(,. 

Frequencies ^nd percentagej of scores. One hundred forty-one (141) 
students responded to Prompt 8. The mean for this writing task was 4.34, with 
a standard deviation of 1.79. Approximately 75 percent of the students scored 
4 or above on ihis task; 24 percent achieved a ranking of 6 or above, while 4.3 
percent achieved the highest »^nking of 8. Table 23 provides a summary of the 
percentages and frequencies at each ranking. 



TABLE 23 


GRADE: 8 PROMPT 


: 8 CONO: 


HOLISTIC SCORE | 

1 








CUM 


VALUE 


FREQUENCY 


PERCENT 


PERCENT 


0 


6 


4.3 


4.3 


2 


15 


10.6 


14.9 


3 


14 


9.9 


24.8 


4 


42 


29.8 


54.6 


5 


29 


20.6 


75.2 


6 


17 


12.1 


87.2 


7 


12 


8.5 


95.7 


8 


6 


4.3 


100.0 


TOTAL 


141 


100.0 
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TABL'E 24 GRADE: 3 SCORES BY DISTRICT 





CVERAlIi 


uisTsncr 1 


DISDUCr 2 


DISIRICT 3 


DISTRICT 4 


rasQacr 5 




vm 


m 




ma 




HOffT 


HCHPT 


IHittT 


FRCHPT 


orta w/ Hean 
PtBwritijig Std 
N 

v/o Hean 
Ptwiting Std 
N 


Plaoa 

3.53 
1.48 

U6 

3.63 
1.82 
112 


BDSsass 

3.85 
1.56 
112 

3,76 
1.65 
123 


Day 

3.09 
1.56 
99 

4.14 
2.18 
103 


Scaiy 

3,53 
1,66 
99 

3,10 
1,84 
99 


Plac« 

3.41 
1,23 
34 


3.63 
1.61 
35 


Plaaa 

4.19 
1 66 
27 


4.27 
1.68 
30 


Day 

3,27 
1,14 

30 


Scazy 

3,62 
1,48 
39 


toy 

4,79 

2,09 
29 


Soaxy 

3,93 
1.34 
30 


Flaoa 

3,29 
1*38 
28 


3,07 
1,U 

27 



3.58 3,80 3,62 3.31 

1.65 1,60 1,95 1,76 

228 235 202 198 



OONDITTOW 

Oral w/ 
PTBwritijig 



Oral v/o 



Std 
N 

Hean 

Std 

N 



DISDUCr 6 


DISTWCT 7 


cnsnacr i 


DisnacT 9 


DISIRICT 10 


msBdCT 11 


m 




TO 




RCHFT 


HCHPT 






Plaot 

3.50 
l.<2 
26 


3.52 
1.17 
21 


toy 

2.87 
1..4 

23 


Scazy 

3.45 
2.04 
20 


toy 

3.81 
2.33 
26 


Scary 

2.35 
1.23 
23 


Flaoa 

3.88 

l.U 

25 


5.13 
1.70 

32 


Placsa 

2,74 
1,98 

31 


4,22 
1,34 
32 


to/ 

3,10 
1,41 

21 


Scazy 

2,89 
1.91 
18 







DISnUCT 12 


DISIRICT 13 


DISIRICT 14 


DISIRICT 15 


DISTRICT 16 






TO 


m 


noffT 


FRCHPT 




PRT'T 


OCWDmOH 




toy 


Scary 


Flaoa 




Plaoa 




Day 


Scary 


toy 


Scazy 


Oral w/ 
Prewriting 


Haan 

Std 
N 


3.12 
1.88 
25 




3,59 
2,01 
29 






3,24 

1-70 
29 


3-06 
1.75 
25 






1,71 
1,23 
21 


Oral v/o 
Ptwriting 


Hean 

N 




4,25 
1,64 
28 




3,07 
1,10 
29 


4-21 
1,75 
26 






2,05 
1.61 

22 


3.45 
2.42 
20 
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TABLE 25 



9^ 9 

gt ^agi rwiations, 
tv PlxwPt and C. mriition 



and H« for Holistic SC30« 



GRADE: 8 



SCORES BY DISTRICT 







1 


Ow 


nil 




DUtrict 1 


District 3 


District 4 


District 5 


District 6 






onl V/ 
Ptwnting 


Oral v/o 
PRwritJng 


Silent V/ 
Ptwriting 


Sil«it w/o 
Prtwrlting 


Ond V/ 
Pzwriting 


SUtnt v/o 
Prwriting 


SUant V/ 

Pmoriting 


SUtnt v/o 
Pxwrlting 


SUant v/o 


oral V/ 


SilOTt V/O 

FKwriting 


ond v^ 
Fkwrltiiq 


Silfflt V/O 
FMwritijig 




Mean 

Std 
N 








3.7S 
1.5t 
IM 




3.40 
1.14 
5 




4.25 
4.«8 
li 


4.00 
1.36 
15 




4.20 
2.28 

5 




5.57 
1.51 
7 


l^3ssession 


Mean 

Std 

N 


4.27 
1.72 
114 


3.44 

1.51 
70 


4.^2 
1.5t 

79 


4.07 
1.49 

150 


3.70 
1.20 
27 


3.83 
1.33 
• 


5.21 
1.23 
28 


4.50 
0.94 
14 


5.15 
0.90 
13 


3.41 

32 


4.00 

0.71 
5 


2.M 
I 28 
45 


4.71 
1.50 
7 


Kisic 


Mean 

Std 

N 








3.59 
1.67 
140 




3.00 
1.00 
5 




4.14 
1.83 
14 


4.42 
1.08 

12 




3.25 
O.H 
4 




5.86 

1.46 

7 


Snakes 


Mean 

std 
N 








4.39 
1.79 
141 




4.60 
1.34 
5 




4.75 
1.71 
U 


4.79 
0.97 
14 




5.00 
2.16 
4 




5.71 
1.50 
7 






4.27 
1.72 
114 


3.44 

1.51 
70 


4.92 

i.sa 

79 


3.95 
1.65 
599 


3.70 
1.20 
27 


3.71 
1.27 
21 


5.21 
1.23 
28 


4.39 
1.S5 
56 


4.57 
1.16 
54 


3.41 
1.07 
32 


4.U 
1.64 
18 


2.96 
1.28 
45 


5.46 
1.48 
28 







District 7 


District 8 


DisU-ict 9 


District 10 


District 12 


District U 


District 14 


District 15 


District 16 1 




Slloit V/ 
Prswriting 


Silent v/o 

1 Prwriting 


Silent v/o 
Ptwriting 


Oral V/ 
PiwriUng 


Silont v/o 
Fkwritijig 


Oral v/o 
Prav. itin? 


Silvit v/o 
Fnuriting 


SUant v/o 
Pm#ritiji9 


Oral V/ 
Rwriting 


SU«t v/o 
Pkwritiiig 


SU«lt v/o 
PKwritlng 


SUent v/o 


SUent V/ 

Prawriting 


SUent v/o 
Ptewriting 


Utter 


Mean 

std 
N 




2.83 
1.33 
6 


3.27 
1.44 
15 




3.89 
1.27 
9 




3.67 
1.03 
6 


3.38 

1.15 
16 




4.43 


2.M 
1.17 
14 


:.25 
1.83 
32 




2.29 
0.76 
7 


f^tssession 


HMn 

std 

N 


4.27 
1.74 

30 


«. 86 
1.07 
7 


4.15 

1.57 
U 


4.U 

1.52 
32 


5.00 
1.20 
8 


4.32 
1.52 
25 


3.67 
1.37 
6 


3.00 
1.41 
15 


6.45 

1.53 
22 


5.14 
1.95 

7 


3.21 
1.25 
14 


4.U 
1.55 
30 


5.48 

1.47 
21 


3.80 

1.79 
5 


Mjsic 


Mm 

N 




3.43 
1.90 
7 


3.92 
1.62 
12 




3.75 
1.16 
8 




4.00 

2.76 
6 


2.53 
1.68 

15 




3.86 

0.90 
7 


2.36 
1.50 
14 


3.57 
1.43 
30 




3.14 
1.07 
7 




Mean 

Std 
N 




4.17 

2.;i 

6 


4.40 
1.84 

10 




4.78 

0.97 
9 




4.17 
0.75 
6 


3.U 
1.53 
15 




6.00 
1.41 

7 


2.33 
1.07 
12 


4.55 
1.90 
29 




5..0 
0.84 
5 


^ J u 




4.27 
1.74 
30 


3.31 

i.ei 

26 


3.88 

1.61 
50 


4.13 
1.52 
32 


4.35 

1.23 
34 


4.32 
1.52 
25 


3.88 

1.57 
24 


3.02 

1.53 
61 


6.45 

1.53 
22 


4.86 
1.58 
28 


2.70 
1.28 
54 


4.12 

1.71 
121 


5.48 

1.47 
21 


3.46 
1.50 
24 
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VI. Discussion of Findings 

Reviewing the results of an assessment such as this one often raises more 
questions than it provides answers. The review is doubly difficult in this 
"ase, since the assessment was a pilot and therefore has a limited population 
and no base-line data for comparative purposes. However, some useful 
information has emerged from the assessment which would seem to have 
implications for future large-scale testing in writing. For the purposes of 
brevity and clarity, the discussion of this information will focus primarily on 
several key issues: the writing prompts, the scoring procedures, and special 
issues such as third readings and gender performance. 
Prompts 

Design. Designing effective prompts is not an easy task. Participants in 
the UWP Summer Institute, all experienced writing teachers, found that what 
they initially thought would be clear and focused writing tasks turned out not 
to be so. Much of the difficulty in designing prompts arose from the 
assumptions made about prior knowledge and skill levels; teachers also 
discovered that directions had to be very explicit to draw a particular kind of 
writing from students. 

The general strategy used in developing prompts was to focus on 
"universal" experiences with which students could identify. Since students 
would not have seen the writing task in advance and would have only 45 minutes 
to complete the task, it was important that the prompt be what Gordon Brossell 
(1987) calls "content fair"; that is, students should be able to select their 
own subject matter. In one case, however, the decision was made to use what is 
called "controlled data" in an effort to see what, if any, difference this 
might make. In a "controlled data" prompt (see Prompt 8 for the eighth grade), 
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the rhetorical situation is carefully defined and the actual information to be 
used in the writing is provided. The student's task, therefore, is to decide 
upon the best approach to applying the information to the writing task. (For a 
useful discussion of designing writing prompts, consult Ruth and Murphy, 1987). 

Originally the design for the assessment called for testing all prompts 
under four conditions, two of which called for silent reading. A small field 
test of one of tht early prompts in a third grade setting quickly repealed that 
third graders, at least those at the beginning of the year, could not handle 
the silent reading effectively in the allotted time. For that reason, third 
graders were tested only under Conditions 1 and 2, both of which had the 
teacher doing oral reading of the prompt. 

Another issue that surfaced during the design stage was the role of the 
teacher. Since the assessment was designed to permit classroom teachers to 
administer it, the directions had to be as clear and as consistent as possible. 
This did not prove to be a difficult task when developing Condition 2, since 
the teacher under this condition simply read the prompt aloud to the students. 
However, under Conditions 1 and 3, where teachers were involved with directing 
the prewriting sequence, very precise directions had to be developed to insure 
consistency at test sites. Many teacliers who administered the assessment under 
Conditions 1 or 3 commented about the script quality of the directions (See 
Appendix F) but in no case did the teachers indicate that the script was a 
major problem in the test administration. 

Effects of Prewriting . 

One :f the key elements to be tested in the assessment was the impact of 
prewriting. All third grade prompts were tested with and without prewriting 
sequences; in all cases where prewriting was used, the teacher directions and 
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the actual prewriting steps were identical (See Appendix H). At grade 8, 
prompt 6 was tested under four conditions, two with prewriting and two without 
prewriting. 

In the cases where prewriting was involved at the eighth grade (Conditions 
1 and 3), the mean for performance was substantially higher than the mean for 
performance witho.t prewriting (See comparison of means in Appendix H). At the 
third grade level, however, the picture was not as clear. With two out of four 
prompts, the means for performance under the prewriting condition were lower 
than those without prewriting. 

A cursory review of the prewriting or planning sheets for third graders 
suggested that far less material appeared on those sheets than was th? case for 
the eighth graders. One might conjecture that third graders have, 
understandably, had far less practice with prewriting (see Graves, 1983) and 
therefore are less accustomed to using it in their writing. In fact, for some 
it might actually prove counterproductive, since in some cases, at least, 
students who had substantial writing on the prewriting sheet often did not have 
a very lengthy final text. 

In further support of this hypothesis, during the initial field testing of 
one of the prewriting sequences for third graders it was noted that the teacher 
did such a thorough job of taking students through the prewriting steps that a 
number of students lost interest in converting their material from the 
prewriting page into the fi.ial draft. At best, then, the success of prewriting 
at the third grade level seems mixed; at the eighth grade level, however, it 
clearly had a positive impact. 
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Most E^^fective Prompt . 

In trying to determine the most effective prompt at each grade level, 
three measures were used: the percentage of unscorable papers a^^nerated in 
connection with the prompt; the means for each prompt, and the percentage of 
papers at the highest level. The most effective prompt, therefore, would be one 
that permits the most able student to do well while also distributing papers 
among the various score points. 

Prompt 2 (Possession) emerged at the third grade level as the most 
effective, followed by Prompt 3 (unusual day). The least effective was Prompt 4 
(scary experience). ,H the eighth grade level. Prompt 8 (Snakes), followed by 
Prompt 6 (Possession), was clearly the most effective; the least effective was 
Prompt 7 (Music). 

The overall distribution among means and scoring percentages for all 
prompts was remarkably close; still, there were also some clear differences. 
In analyzing the characteristics of the effective prompts at both grade levels, 
one might conjecture that a shared characteristic was the concretPiiess found in 
both; that is, in Prompt Z, studen s were asked to select a prized possession. 
Although some students selected somewhat abstract choices, such as family, mos". 
chose an object which clearly did have an important role in their lives; the 
result was obvious--the writing was more specific, generally oetter organized, 
and the writers displayed more of their personality. Prompt 8 also was 
concrete but in a different way; the presence of the controlled data, and the 
specific rhetorical situation provided a clear direction for students, and they 
seemed to respond well. Remarkably, this prompt was the only one in the entire 
assessment which did not have any third readings (See Appendix I). 

The two prompts identified as least effective. Prompt* 4 a-d 7, do not 
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appear to have any element, in common. Prompt 4 asked students to relate a 
scary experience; the emphasis was placed on personal experience but a 
substantial percentage of students chose to construct stories of fantasy and 
horror, leading to a fairly high percentage of unscorable papers; in addition, 
there were fewer papers at the high end of the scoring ranks. Readers during 
the holistic scoring sessions commented on the difficulty students seemed to 
have relating a believable narrative to emotion. With Prompt 7, the problem 
seemed to be a lack of familiarity with strategies for persuasive argument even 
with the audience identified. Although the topic of rock music seemed to 
stimulate some strong feelings on the part of the writers, many exhibited 
considerable difficulty in marshalling any convincing evidence or a consistent 
argument. 
Special Issues 

Several special issues emerged from the assessment. Among these were the 
relationship of third readings to the overall scoring process, the percentage 
of unscorable papers, the performance of m,ales versus females, and the 
comparison of performance by third graders and eighth graders on the same 
writing prompt. 

Third Readings . 

In holistic scoring, if readers disagree by more than one point on a 
particular writing sample, then the piece is read by a third reader. Myers 
(1980) and Duke (1985) indicate that an inter-rater reliability coefficient of 
,80-. 90 is reliable in holistic scoring. In this assessment, an overall inter- 
rater agreement of .94 was achieved in scoring 1725 writing samples. 

At the third grade level, the overall inter-rater re'liability coefficient 
was .96. The greatest number of third readings {23 out of 862) occurred with 

81 

ERIC 



those samples written under Condition 2--teacher oral direction without 
prewriting. Prompt 2 (Possession) under Condition 2 was the single prompt to 
attract the most third readings (12 out of 123). Prompt 1 (Place) under 
Cor.jitlon l--teacher oral direction with prewriting--garnered the fewest third 
readings (1 out of 116). 

At the eighth grade level, the overall inter-rater reliability coefficient 
was .93. Prompt 7 (Music) under Condition 4 attracted the most third readings 
(14 out of 14b;. Prompt 8 (Snake), tested only under Condition 4, had the 
distinction of being the only prompt in the full assessment not to garner any 
third r;^adings. (See Appendix I for comparisons among prompts). 

Unscorable Papers . 

The category of unscorable papers is reserved for those writing samples 
which clearly do not address the topic, are blank or illegible. At Grade 3, 6.6 
percent (57 out of 863) were unscorable. At Grade 8, 1.7 percent (15 out of 
862) were unscorable. Prompt 4 at the third grade level produced the most 
unscorable papers (10.1 percent). The majority of these were off topic, with 
students choosing to create fantasy rather th,^n to deal with real life 
experiences. Prompt 8 at the eighth grade level produced the most unscorable 
papers (4.3 percent); most of these seemed unable to deal with the coi.trolled 
data and/or specified rhetorical situation. Table 26 shows the percentages of 
unscorable papers for each prompt. 
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Table 26 Percentage of Unscorable Papers by Prompt 


Prompt No. 


Percentage 


1 


5.7 


2 


2.6 


3 


8.9 


4 


10.1 


5 


.6 


6 


.7 


7 


3.4 


8 


4.3 



Gender . 

Differences between male and female performances at both the third and 
eighth grade were consistent. Females at both grade levels outperformed males 
at the higher score points of 6 or above (10 percent for the third grade, 25 
percent for the eighth grade). More males than females at both grade levels 
scored 3 or below (53 percent at the third grade, 49 percent at the eighth 
grade). The percent of unscorable papers for males and females at both grade 
levels showed no real difference. In terms of mean performance, in all cases 
except one, the mean for females was higher at both grade levels than it was 
for males. 

The third grade population of 861 students contained 461 males and 400 
females. An examination of their overall performance shows that approximately 
38 percent of the females scored 3 or lower while 53 percent of the m^les 
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registered scores of 3 or lower. It should be noted, however, that the percent 
of unscorable papers for males and females remained virtually identical (6.5 
percent for females, 6.7 percent for males). At the higher score ranges, 6 or 
above, the females showed a higher percentage of achievement (18 percent) than 
the males did (8 percf-nt). Of the third graders receiving a score of 8, 82 
percent were females. 

Males performed best on Prompt 3 under Condition 2; poorest performance 
occurred on Prompt 4 under Condition 2. For the females, the average mean on 
all writing tasks was 3.84 with a standard deviation of 1.74. The females 
performed best on Prompt 3, Condition 2, while their poorest performance 
occurred also on Prompt 3 under Condition 1. (See Appendix I for comparison of 
means on all writipo tasks). 

The eighth grade population of 861 students contained 373 males and 488 
females. The average mean for male eighth graders on all writing tasks was 
3.57 with a standard deviation of 1.49. For the females, the average mean on 
all writing tasks was 4.46, with a standard deviation of 1.59. An examination 
of their overall performance shows that approximately 26 percent of the females 
scored 3 or lower, while 49 percent of the males registered scores of 3 or 
lower. The difference in unscorable papers was minimal (1 percent for females, 
2 percent for males). At the higher score ranges, 6 or above, the females 
showed a higher percentage of achievement (25 percent) than did the males (12 
percent). 0. the eighth graders receiving a score of 8, 7S percent were 
females. 

Males performed best on Prompt 6, Condition 3, while their poorest 
performance occurred with Prompt 7 under Condition 4. The females performed 
best on Prompt 6, Condition 3, while their poorest performance was recorded 
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also with Prompt 3 under Condition 2. {See Appendix I for comparison of means 
on all writing tasks). 

Performance on the Same Prompt . 

Only one prompt was shared in common between the two grade levels. Prompt 
2 at the third grade level and Prompt 6 at the eighth grade level were 
identical. It is possible to get a picture of how third and eighth graders 
handled this prompt only under Conditions 1 and 2. 

Some interesting comparisons emerge when the performance of the two grade 
levels is placed side by side. At both grade levels, the mean was higher under 
Condition 1 and the percentage of students scoring 2, 3 or 4 under that 
condition was lower. Aside from these commonalities, however, the rest of the 
picture is somewhat less clear. For example, for the eighth grade under 
Condition 1 there were no unscorable papers while at the third grade. Condition 
1 registered a few unscorable papers. In terms of percentages of high scores 
(7-8), Condition 2 produced better results at the third grade while at the 
eighth grade, no difference existed between the two conditions. Finally, this 
prompt produced the most effective writing for the third grade and was second 
most effective for the eighth grade. 



ERIC 



85 



VII. ANALYSIS OF WRITING ERRORS 

As context for this analysis of writing errors, it is important to 
remember two important points about the pilot test of the Utah Writing 
Assessment: 1) the early October test represents an assessment of students 
after two and seven yaars of schooling respectively, i.ot three and eight years; 
2) students had little opportunity to revise, edit, and proofread their work 
during the test period of 45 minutes. In other words, the pilot test 
represents an assessment of "first draft" writing for students just entering 
grade 3 and 8. 

To analyze errors, a decision was made to examine ten papers from each of 
the four score points at each grade level --a total of 80 papers. For sampling, 
papers were pulled randomly, then scanned for gender code. Next, additional 
papers were pulled so that an equal number of boys and girls were included at 
each of the score points for both grade levels. 

The purpose of this analysis was to provide an overview of erro.- 
frequencies that characterize writing at differing le"els of achievement. A 
simple taxonomy was developed so that frequencies of errors could be tabul.ted 
systematically. This taxonomy is shown below: 
I. Punctuation (sente ce end stop) 

A. Fragment 

B. Comma splice/fused sentence 

II. Usage 

A. Subject/verb agreement 

B. Capitalization 

III. Spelling 

A. Homonyms/contractions 

B. General spelling 

These categories of error were derived from cor. standards and from 
rubrics developed by the National Assessment of Educational Progress. (For 
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students at grade 3, frequencies of error were also tallied for a third 
category under the Punctuation heading--the "on-and-on sentence.") 

For each grade level, the UWP co-directors read ten papers at each of four 
score points (8, 6, 4, and 2), tallying errors and counting total words per 
paper. Dividing the total errors in each category by the total number of words 
at each score point provided a way of describing error rate per 1000 words. 
Error rates are, in effect, probability statements-the number of items of a 
certain type that one would expect to encounter in 1000 words of text at a 
given score point. 

Table 27 provides an overview of error rates per 1000 words at each of 
four score points for grade 3. In examining these data, it is important to 
note that students at scorepoint 8 wrote over twice as many words (mean length 
= 131 words) as students at scorepoint 2 (mean length - 64 words). In other 
words, fluency seems to be a very important factor in qualitati' i ratings (See 
Appendix J for frequencies of error). 



TABLE 27: GRADE 3 ERROR RATES PER 1000 WORDS 



I. Punctuation (sentence end stop) 


SCORE 8 


SCORE 6 


SCORE 4 


SCORE 2 


A. Fragment 


.005 


.008 


.003 


.005 


B. Comma splice/fused sentence 


.018 


.017 


.023 


.034 


C. On-and-on sentence 


.002 


.008 


.010 


.012 


II. Usage 










A. Subject/verb agreement 


.002 


.002 


.002 


.000 


B. Capitalization 


.0i3 


.022 


.015 


.042 


III. Spelling 










A. Homonyms/contractions 


.006 


.008 


.009 


.003 


B. General spelling 


.069 


.082 


.090 


.146 
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There are at least three points of interest in Table 27. Note first of 
all that error rates tend to increase as qualitative scores decline. Th^> 
pattern seems especially clear for general spelling. Note also the very l(»w 
rates of error for sentence fragments and subject/verb agreement. Note finally 
that with the "on-and-on sentence," students at scorepoint 2 produce about six 
times as many errors as their counterparts at scorepoint 8. As expected, 
students at lower developmental stages are less in control of basic writing 
mechanics at grade 3. 

Table 28 provides the same comparative information for students at grade 
8. However, there was no need to tally the "on-and-on sentence" for eighth 
grade papers. As was the case with third graders, fluency for students at 
scorepoint 8 (mean length = 257 words) nearly doubled that for students at 
scorepoint 2 (mean length - 137 words). (See Appendix J for frequencies of 
error) . 



TABLE 28: GRADE 8 


ERROR RATES PER 1000 WORDS 






I. Punctuation (sentence end stop) 


SCORE 8 


SCORE 6 


SCORE 4 


SCORE 2 




A. Fragment 

B. Comma splice/fused sentence 


.001 
.010 


.003 
.011 


.003 
.014 


.001 
.012 




II. Usage 












A. Subject/ verb agreement 

B. Capitalization 


.001 
.002 


.001 
.002 


.oon 

.002 


.001 
.009 




III. Spelling 












A. Homonyms/contractions 

B. General spelling 


.001 
.010 


.004 
.016 


.010 
.037 


.014 
.028 
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Table 28 holds a few surprises. Note first of all that error rates in 
three categories (fragments, comma splice/fused sentence, and subject/verb 
agreement) are virtually identical across score points. Note secondly the 
extremely low incidence of error in fragments and subjpct/verb agreement, a 
pattern already noted in grade 3 data. Note finally that with spelling errors, 
there are clear differences between students at scorepoint 4 and 2 as compared 
with their counterparts at scorepoints 8 and 6. For eighth graders at least, 
spelling seems to be the error type that distinguishes low-rated papers from 
high-rated ones. 

A comparison of error rates for grades 3 and 8 is shown below in Table 29. 
This table provides a general picture of the decline in error rates over time 
(rive years of schooling). 



TABL^ 29- A COMPARISON OF GRADE 3 AND GRAPE 8 


1 ERROR RATES PER 


1000 WORDS 




SCORE 8 


SCORE 6 


SCORE 4 


SCORE 2 


Punctuation (end stop) 


GR3 


GR8 


GR3 


GR8 


GR3 


GR8 


GR3 GR8 












Fragment 


.005 


.001 


.008 


.003 


.003 


.003 


.005 .001 


Comma splice/fused sentence 


.018 


.010 


.017 


.011 


.023 


.014 


.034 .012 


Usage 
















Subject/verb agreement 


.002 


.001 


.002 


.001 


.002 


.000 


.000 .001 


Capital ization 


.013 


.002 


.022 


.002 


.015 


.002 


.042 .009 


Spell ing 
















Homonyms/contractions 


.006 


.001 


.008 


.004 


.009 


.010 


.003 .014 


General spelling 


.069 


.010 


.082 


.016 


.090 


.037 


.146 .028 


From the data presented 


in Table 29, it 


seems 


clear 


that 


students in Utah 



make progress in virtually all areas as they progress through the grades. 
Progress is evident at each of the four score points, not just at the upper 
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levels of development. In certain areas, such as capitalization and general 
spelling, progress is very dramatic; in other areas, such as comma splice/fused 
sentence, declines in the rate of error are slower, often persisting into the 
high school grades. For students at scorepoint 2 in the eighth grade, certain 
types of error (spelling of homonyms/contractions) actually increase slightly. 

There are possible implications for instruction in these data. Consider, 
for example, the surprisingly low incidence of error in certain areas--such as 
sentence fragments and subject/verb agreement. Since th»se errors appear to be 
well under control, even at grade 3, they probably deserve relatively less 
attention from language arts teachers. Likely target areas would appear to be 
comma splice/fused sentence, capitalization, and general spelling. 

The final table presented here relates to fluency--the mean number of 
words produced by students at each of the four score point for grade 3 and 
grade 8. These data are shown in Table 30. 



TABLE 30: MEAN FLUENCY FOR GRADE 3 AND GRADE 8 

SCORE 8 SCORE 6 SCORE 4 SCORE 2 

GRADE 3 131 92 106 64 

GRADE 8 257 313 220 137 



Since better students write more fluently, it seems clear that fluency itself 
should be a high instructional priority. Without fluency, of course, one 
cannot make progress in basic skills. 

Aw.cording to the National Assessment of Educational Progress--an 
assessment of student writing at ages 9, 13, and 17--the number of errors that 
students make in punctuation, capitalization, and fragments does not appear to 
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decrease but remains more or less constant with increasing age. This occurs 
not as result of student perversity but because students generate more text and 
thus produce more opportunities for error. Indeed, in attempting '..ore complex 
syntax and more complex forms of discourse, students often increase their rates 
of error. In her landmark study. Errors and Expectations (1977), Mina 
Shaugnossey observed, for example, that misuses of the semicolon become 
"epidemic" when first introduced. 

A sensibla perspective on error is based on an understanding of children's 
language development. As Graves (1983) and Calkins (1987) have shown, many 
errors disappear naturally as students engage in meaningful reading/writing 
activities-writing regularly for real communicative purposes. Children should 
attend to error at the final stages of the writing process, not in its 
beginning stages. With regard to writing development, we should remember that 
the process of skill acquisition is glacially s^ow and that direct attention to 
error through grammatical analysis, worksheets, and other non-writing 
activities often produces deleterious effects on student writing. From roughly 
80 years of empirical research, we know that the direct study of grammar does 
not help students to write well (Hillocks, 1986). 
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VIII. RECOftlENDATIONS 

In preparing this report for the Utah State Office of Education, we 
have been keenly aware that, to some extent, assessment drives curriculum. In 
Utah, as elsewhere, teachers pay attention to what is tested and act 
accordingly in the planning of day-to-day lessons and the shaping of programs 
for children. 

Because of the political nature of assessment, we believe that testmakers 
have a special ethical responsibility to the community. Assessments must be 
designed to reinforce and support sound educational practice, not undermine it. 
Moreover, assessments must be undertaken with the knowledge and consent of 
educators at all levels. Finally, assessment results must be interpreted 
thoughtfully and used in an educationally responsible way In short, a sound 
state assessment must itself be continually assessed in relation to the 
standards articulated above. 

From our association with educators in other states, we know that there 
are graat dangers for abuse of assessments in writing. In Florida, for 
example, assessments have been designed that actually promote skill -and-drill 
teaching. In some states, teachers have been virtually excluded from the 
policy making process that determines the nature of assessment. In still other 
states, particularly in the southeast, district administrators have used 
assecsments of student performince to compare teachers for salary decisions, 
and certain newspapers have made invidious "rankings" of schools. 

The net result of such abuses, of course, has been a profound 
demoralization of teachers and an attendant deterioration of educational 
quality. In those parts of the country where legislators and policy makers use 
test scores as a public relations weapon to make teachers "accountable," many 
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teachers simply "teach to the test " with little regard for their larger 
responsibilities. In an effort to bring up test scores, other teachers put 
undue pressure on children and end up creating more literacy problems than they 
solve. 

In Utah, whe^-e the political climate demands increased "productivity" 
because or financial constraints, a strong potential exists for abuse of 
assessments. Such abuse, we believe, would arise not from any malevolence on 
the part of legislators or policy makers but simply from the belief that 
educational reform (a desirable end) can be effected through coercive or 
manipulative strategies. In short, we believe that even the best of intentions 
may be perverted as prog.ams of assessment are implemented. 

Clearly, the alternative approach is one that involves teachers from the 
outset. We believe that a coherent plan for assessment should be developed and 
that teachers should be involved in approving it. In our opinion, a Writing 
Assessment Board should be created to oversee all aspects of the assessment 
process. This Board would be a funded entity, with authority and 
responsibility for conducting writing assessments in Utah. A major function of 
this Board would be to involve teachers throughout the state in assessment and 
to prepare reports for the State Office of Education. 
General Recommendations 

Having expressed our concerns, we respectfully invite consideration of the 
following general recommendations. 

1. Create a Writing Assessment Board, perhaps through the Utah Writing 
Proje. to oversee the overall plan for assessment as well as its 
implementation. Working with state officials, this Board would 
determine assessment policy, guard against potential abuses of 
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assessment data, implement a statewide writing test, and prepare 
assessment reports. Teachers would have a strong voice on the Board. 

2. Use writing assessment as a staff development tool for English/language 
arts teachers. The Writing Assessment Board would conduct regional 
scoring sessions, with Utah teachers serving as paid scorers. This 
approach would familiarize teachers with the goals of the assessment, 
its standards, and its relation to the core curriculum. Teachers, in 
turn, would be able to provide a steady flow of on-the-SDOt feedback to 
the Writing Assessment Board. 

3. Develop materials aimed at classroom teachers that would assist them in 
their work with children. The Writing Assessment Board would develop 
model lessons and/or staff development videotapes showing teachers how 
to implement core curriculum standards and thus prepare for the 
assessL.jnt. These materials could be disseminated through the Utah 
Writing Project summer institute program as well as through its network 
of teacher/consultants. 

4. Conduct inservice sessions at the district level, using papers scored 
in the assessment. The Wricing Assessment Board would help school 
officials understand areas of strength and weakness in district 
profiles. With local papers at hand, teachers could then be helped to 
look beyond holistic scores to diagnostic and instructional 
implications. Such an approach would also drive home the value of 
portfolio assessment in district programs. 
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Specific Issues 

Beyond these general policy recommendations, certain specific issues 
emerge from the pilot test of the Utah Writing Assessment. These are 
enumerated below, with accompanying recomitier.dations where appropriate. 

1. If a decision is made to assess grade 3 students, the only reasonable 
time for assessment is in the spring. Many students at grade 3 are not 
sufficiently mature to handle the demands of testing early in the 
school year. If a program of autumn testing is implemented, grade 4 
students should be assessed, not grade 3. We reconrniend that serious 
consideration be given to testing of grade 4 (or grade 5) rather than 
grade 3. 

2. Spring testing reveals the effects of teaching for several months; 
however, autum testing would presumably enable teachers to emphasize 
writing during the remainder of the school year. Assuming relatively 
speedy turn-around on score reports and diagno5;tic work with papers at 
a local level, the autumn testing schedule would make instructional 
sense. If a program of spring testing is initiated, it would seem 
reasonable to test at grade 3, 7, and 11; 4, 7, and 10; or 5, 8, and 
11. If testing is done in the fall, it might be done at 4, 7, and 10; 
4, 8, and 12; or 5, 8, and 11. (Each of these sequences provides for 
an equal number of years between assessments.) Such a program would 
provide assessment information to all of the schools in Utah. The 
decision about whether to test during the spring or fall months must be 
based on the purposes for assessment and the financial commitment of 
the State Office to assist local districts. 
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3. At least two prompts should be used at the upper levels of assessment 
(middle school and high school levels). Multiple prompts will help to 
ensure that schools attend to all standards in the core, not just ones 
that are traditionally tested on the assessment. 

4. Prewriting sequences should be built in where appropriate on prompts 
and teachers should have scripted materials to use in working wUh 
their classes. 

This prewriting must be brief, however, to provide plenty of time for 
drafting and proofreading. 

5. leacher-reaci (oral) directions, with students following along, should 
be used in the elementary grades; (silent) reading of directions at the 
upper levels of assessment is appropriate. A summary of key findings 
from the pilot test of the Utah Writing Assessment is found in the 
Overview of this document (pp. 4-5). 
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APPENDIX A: 



Distarict 
AUPINE 



PILOT TEST SCHOOLS 1987-1988 

PIUDT TEST SCHOOIS 1987-88 



Si5)erintendent 



Principal— Elesmentary 



Principal—Seoorriary 



EEBWHt 



Clark Oqx 

Alpine School District 
American Fork, UT 84003 
Tele: 756-9671 



I^nn Haslem 

Beaver School District 

290 N Main 

Beaver, UT 84713 

Tele: 438-2291 



Stan Haxward k-6 
Central School 
95 N 400 E 

Pleasant Grove, UT 84062 
Tele: 785-3121 

Enrollneit: 500 



Kay Giles 

Qrem Jr High School 
765 N 600 W 
Qrem, UT 84057 
Tele: 225-1700 

Dirollment: 974 



7-9 



Val Smith 
Belkne^ School 
Bean^, UT 84713 
Tele: 438-2281 

Enrollinent: 463 



K-6 



mivjivB Darzell White 

Box Elder School DL«rtrict 
230 W 200 S 

Bric^iain city, UT 84302 
Tele: 723-5281 



cram 



Clark Puffer 
Cache School District 
2063 N 1200 E 
Logan, UT 84321 
Tele: 752-3925 



Donald Potter 
Garland School 
250 N Main 
Garland, UT 84312 
Tele: 257-3129 

Enrollment: 335 



K-6 LaMar Bourne 7-8 

Box Elder Jr High School 
18 S 500 E 

Brigham City, UT 84302 
Tele: 723-3493 

Enrollment: 777 



dair Larkin 
Wellsville School 
90 E 100 S 

Wellsville, UT 84339 
Tele: 245-3764 



K-5 Holly Petersen 6-8 

Spring Creek Middle School 
350 W 100 N 
Ptxvidenoe, UT 84332 
Tele: 753-6200 



Enrollment: 591 



Bailment: 




Ell B. Sorenscxi 
Carixsi School District 
65 E 400 N 
PO Box 1438 
Price, ur 84501 
Tele: 637-1732 



John B. Angotti 
Castle Heists School 
750 N Homestead Blvd 
Price, ur 84501 
Tele: 637-7177 

Enrollment: 497 



K-6 Tonita Crookston 7-9 

Mont Harmon Jr High School 
60 W 400 N 
Price, ur 84501 
Tele: 637-0510 

Enrollment: 739 
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District 



Si^Derintendent 



Principal—Elementary 



Principal—Seoandary 



Richard Kenclall 
Davis School District 
45 East State Street 
Farmington, UT 84025 
Tele: 451-1251 



Forest Barker 
Adams School 
2500 N 2200 E 
Layton, UT 84041 
Tele: 546-7914 



K-6 Ross Poore 7-9 

Central Davis Jr Hic^ School 
663 Church Street 
Layton, OT 84041 
Tele: 546-7923 
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Enrollment: 764 



EnrolLnent: 1,075 

Carol Greenwood 
Syracuse Jr. High 
1450 S 2000 W 
Syracuse, UT 84041 



Demis E. Nelson 
Emery Schc»l Dlstxlct 
130 N Main 

Huntingtxin, UT 84528 
Tele: 687-9846 



Morris Mower 
Ferrcn School 
Ferron, UT 84523 
Tele: 384-2383 

Enrollment: 513 



X-6 Sanuel Singleton 7-9 

San Rafael Jr High School 
Perron, OT 84523 
Tele: 394-2335 

Enrollment: 351 



o 
o 



Jctm Reed call 
Granite School District 
340 E 3545 S 

Salt lake City, UT 84115 
Tele: 263-6100 



Dale Hartzigson 
Jerry D. Pulsipher 
West Keams School 
4900 S 4620 W 
Keams, UT 84118 
Tele: 968-0552 
964-7680 
Enrollment: 1,007 



K-3 
4-6 



Barry Richards 7-9 
Jnc Jr High School 
4495 S 4800 W 

West Valley City, UT 84120 
Tele: 969-6271 

Enrollment: 1,387 



JCBDMt 



Raymcaid Whitteriburg 
Jordan School District 
9361 S 300 E 
Sandy, OT 84070 
Tele: 565-7100 



Kreig Kelley K-5 

Canyon View School 

3050 E 7800 S 

Salt Lake City, UT 84121 

Tele: 565-7436 



J. Paul Xbchevar 6-8 
Indian Hills Middle School 
1180 E Sanders Road 
Sandy, OT 84070 
Tele: 565-7526 



Enrollinent: 819 



Enrollment: 1,502 



11 



District 



Si:perintendent 



Principal—Elementary 



Principal — Secondary 



NEBO 



James Blair 

Logan city School District 
101 West Center 
Logan, Ur 84321 
Tele: 752-1811 



J. Wayne NelscMi 
Nebo Sdxjol District 
350 S Main 

Spanish Fork, UT 84660 
Tele: 798-8651 



Larry Jaoobsen 
Adams School 
530 N 400 E 
Logan, UT 84321 
Tele: 752-2275 

Ekirollinent: 488 



K-5 Raymond Haslam 

Mt. Logan Midiile School 
875 N 200 E 
Logan, ITT 84321 
Tele: 752-4755 

E^irollment: 940 



Ten Hudson 
Santaquin School 
25 S 400 W 
Santaquin, ITT 84655 
Tele: 754-3611 



6-8 



K-7 Lynn Jones 

Payson Jr Hic^ School 
1025 S Highway 91 
Payson, UT 84651 
Tele: 465-9231 



8-9 



James Vfest 

Ogden School District 
2444 Adams Avenue 
OcfJen, ur 84401 
Tele: 399-3456 



Ekirollnent: 514 

Send to Robin Fullenbach 



Dale Ihcnpscn 
Polk School 
2615 Polk Avenue 
Ogden, UT 84403 
Tele: 394-3939 



Enrollment: 718 



K-5 Vem Call 6-8 

Mound Fort Middle School 
1396 lAbesctY Avenue 
Ogden, UT 84404 
Tele: 399-3456 



Ekirollinent: 428 



Enrollment: 594 



SMJ lAKE cm 



John Bennion 

Salt Lake City School Dist 
440 E 100 S 

Sedt Lake City, UT 84111 
Tele: 322-1471 



Sally Trost K-6 
Highland Park School 
1738 E 2700 S 
Salt Lake City, UT 84105 
Tele: 481-4833 



Larry Jaisen , 7-8 

Bryant Intemediate Schdbl 
40 S 800 E 

Salt Lake City, UT 84102 
Tele: 533-3008 



Enrollment: 528 



Enrollment: 671 



^ J™t Hal Jensen Kenneth Maughan K-6 Mitchell Kdauli 7-12 

San JUan School District Moittioello School Whitehorse High School 

207 N 100 E PC Box 189 Montezuma Creek, UT 84534 

. . Montioello, UT 84535 Montioello, UT 84535 itele: 651-3427 I 1 J 

1 1 o Tele: 587-2254 Tele: 587-2241 
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EnxolLnent: 409 



Enrollment: 364 



District 



Si:t)erintendent 



Principal—Elementary 



Principal — ^Seocxidary 



SEVIHt 



John lUft 

Sevier School District 
195 E 500 N 
Richfield, OT 84701 
Tele: 896-4406 



Kent Qiristensen 
Saiina School 
210 W 300 N 
Saiina, ITT 84654 
Tele: 529-7462 

Enrollment: 638 
Teacher: Randy Brown 



K-5 Russell PetersOT 

Red Hills Middle School 
496 W Center 
Richfield, OT 84701 
Tele: 896-6421 

Enrollment: 441 



6-8 



M^BR Jay Tacgart Cheryl K-6 John Bushman 7-9 

Weber School District Washington Terrace School Roy Jr High School 

1122 Washington Blvd 125 E 4475 S 5400 S 2100 W 

Ogden, Ur 84404 Ogtoi, OT 84405 Roy, OT 84067 

Tele: 479-8889 Tele: 392-6733 Tele: 825-1605 



Enrollment: 520 



Enrollment: 834 
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APPENDIX B: PILOT TEST CODING SHEET 



prior TEST 

ODDING SHEET 



Alpine 1 Oentxal School 11 1 (35) 

2 2 (35) 

Beaver 2 Belknpt School 2 1 2 (28) 

2 1 (29) 

Box Elder 3 Garland School 3 3 1 (36) 

4 2 (37) 

Cache 4 Wellsville S«. moI 4 3 2 (30) 

4 1 (31) 

Caztxn 5 Castle Heists Sch 5 1 1 (28) 

2 2 (29) 

Davis 6 Adams School 6 1 2 (32) 

2 1 (33) 

Bwery 7 Ferron School 7 3 1 (25) 

4 2 (23) 

Granite 8 West Keams Schorl 8 3 2 (29) 

4 1 (29) 

JortJan 9 Canyon View School 9 2 2 (33) 

Ixjgan 10 Adams Sciiool 10 l 2 (33) 

erIc ' ' 



SEOOHDMCf SCHDOL IK). HOIET COPITEOH 



Qren Jr High 17 2 1 (31) 

(Sept. 25) 1/2,3,4 4 (31) 



Box Elder Jr H S 19 2 3 (33) 

1,2,3,4 4 (34) 
1,2,3,4 4 (35) 



Spring Creek N S 20 1,2,3,4 4 (35) 

1,2,3,4 4 (28) 



Mont Hamcn Jr HS 21 2 1 (32) o 



1,2,3,4 4 (34) 



Central Davis Jr 22 2 2 (35) 

1,2,3,4 4 (35) 

Syracuse Jr HS 34 2 2 (24) 

San Rafael Jr HS 23 2 3 (32) 

1,2,3,4 4 (32) 



JTKJr High Sch 24 1,2,3,4 4 (35) 

1,2,3,4 4 (30) 



Indian Hills MS 25 2 1 (35) 

1,2,3,4 4 (35) 



Mt. Logan 16 26 2 2 (27) 

(Oct. 5) 1,2,3,4 4 (28) 



Dtgimcr IP. ratmmarr smm. MP. noBT 

Nebo U Santaquin School 11 3 

4 



Ogden 



12 Polk School 



12 3 
4 



Salt lake 13 Highland Park Sch 13 l 

Sept. 25 2 



San Juan 14 Montloello School 14 1 

2 



Sevier 



15 Salina School 



15 3 
4 



Weber 



16 Washington Terrace 16 3 

4 



' 'J 
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1 (25) Payscn Jr HS 27 2 3 (33) 

2 (22) 1,2,3,4 4 (31) 



2 (29) Mound Fort M5 28 1,2,3,4 4 (31) 

1 (28) 1,2,3,4 4 (37) 



1 (29) Bryant Intermed. 29 2 1 (32) 

2 (29) 1,2,3,4 4 



2 (30) Mhitehorse H5 30 2 2 (21) 

1 (30) 1,2,3,4 4 (21) 



1 (31) Red Hills K5 31 1,2,3,4 4 (34) 

2 (31; 1,2,3,4 4 (18) 

1,2,3,4 4 (35) 

1,2,3,4 4 (24) 

1,2,3,4 4 (31) 



o 



2 (24) Rcy Jr High Sch 32 2 3 (30) 

1 (23) 1,2,3,4 4 (35) 
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APPENDIX C: TEST ADMINISTRATION MATERIALS 

PILOT TEST: UTAH WRITING ASSESSMENT. 1987 SITE CODE NUMBER 

fimril Pirftftions for Teachers - Please PmH r,r fff"Tiy 

qJSde'^sY a'nd'rSJ t'h°J'..Ah \Vl °/ "''^'^'^ performance conducted at 

graaes J ana 8 by the Utah State Office of Education. In this oilot tP^t oinht 

nr'Jij:? f''^*'^^ ""der varying cond-tions. Te t sho ?d Jo Sivli^Sn 

October 1 or 2, at your option. If you have questions, please call 750 2222 

BEFORE THE ASSESSMENT 

October ] for 2). riere^e some points to discuss w ith vour stuHontc - — 

j;. /5P??^ - On October 1 (or 2) we will be doing some writing. Tne purpose of 
this writing is to show teachers and parents how well Utah studlnts art Zng 

h-f-'^^' 9et directions on what to write about on the dav of the 

test. Then you will have about 30 minutes to develop a good piece of wrUing. 

sav ^hif?^;^.J*' do a good job of writing, you should plan v^hat you're going to 
say, then wrna and then check it over carefully. If you finish earl v vou mav 
read a library book. Please don't talk or disturb others dur!I!g thriritin5 

Jho H^f- ^l"'.* ^®"9th for this writing. Just think about what 

the directions say and do the best yju can in following them. . 

"J.^^^*^^^ - yo" make mistakes in your writing, or if you want to chanae 
something, you can cross through a word and rewrite n'ektly above your crossout ^ 

6. Asgin^ncg If you don't know what to write, raise your hand and I'll helo 
rs^e Ihat^' directions. But I can't help you with the actual writ ng. 11 f 
to see what you can do on your own-without adult help or even a dictionary 

7. Rgfiofijdna. You won't have time to recr.py. So after you've finished writina 
make sure to read over your paper and put in any last-minLte ^hangSs! ^' 

fharoS^^^^i 1°^??^ use either printing or cursive handwriting. Bring a 
sharpened, soft lead (#2) pencil or black pen for the writing test. 

9 Mat^riili. I will give you paper to write on. Use one side of the paper 
only. If you need more paper, let me know and I'll give it to you. 

oL JhnJiy^^nT: Your writing will be graded, but the score doesn't go in my 
nit. wil read your writing. Do your best work so that ol? 

school gets a good score on writing. 

SbeM^SSJ fJr^S^c?'*?'^'" DIBITS OF THEIR PHONE 

for Snnot r^i«t° o f^udents For students who do not have a hone phone 
irstXt,^3 r J- ^ • four-digit number. To make a master list 

?Lsr n/ic H numbers, circulate a sheet on which students write down 

their names and number:. Have this sheet for reference on test day. 

lllZl ulf^^'^F h (or 2), assemble your writing packets, soft lead (#2) pencils, 
master list of student code numbers, and materials for students who finish early. 
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Pilot Test 2 

DURING THE ASSESSMENT 

tlJ^w\1?^vnV^ ^^^^ ^ "^'"^^^ a;;?^PSs.Pnt p nod. Ch.rk thp .t.rt 

Each student should hav.> a .nft I^.h ( *2) pencil nr Mat-, npn Ui th .he c1a«;s 
qathgr^d 9? ? group and desks cleartd-';:^^^. p.^Vh..:, JI^_p^^^^ "'" -^ 

iAmoJi'c^^^^'fi" k'^J be doing some writing as part of a statewide test. In a few 

SSJf; '''r"'^ ''^'.^l"^ P'^'^®*' ^'^'l^ ^ill some numbers before we 

begin writing. For now, listen to these directions. 

2. Write on one side of the page only. Use soft lead (#2) pencil or black pen. 
If you need extra paper or a pencil during the writing, raise your hand. You may 
either print or use cursive for this piece of writing. 

'"^'i® * mistake, or if you want to change something, cross through a 
word and rev.nte neatly abovP your crossout. You won't have time to recopy. so 
do your best work in the 30 minutes or so that you have to write. '*'^"»'J'' 

II;t^^^^?^^"'^^f^^^^^: P^"" •^^^^ * ^'^^^^y (O'^ work on another activity) 
until I tell you that the writing time is over. Don't talk or disturb others. 

ILiJ^ you have any questions during the test, raise your hand. I'll be able to 
help you with directions but not with the actual writing. If you don't know how 
to spell a word, put down what looks right to you. 

?; .J?!^?'"''®'' *° ^° ""I®. P^*""i"g before you begin writing. And it's a good idea 
to check over your writing carefully when you are finished. 

folloSino^Sirpr ll^."^''^^'' "''""""^ i- n mediatplv with tho 

STUOplfT mV°i!liM°prD*'!.® first page of your packet, you see an "A." Fill in your 
STUDENT CODE NUMBER here. Remember, this is the last four digits of your phone 
number. Write down this number after "A." ^ *^ 

If Students cannot remember t h eir code numbers, refer to vour ma^t^r M^t 

8. To the right, you see a "B." If you are a boy, put down a "1." If you are a 
girl, put down a "2." (GENDER CODE) ^ 

9. Now you see "C." Write down the number . (DISTRICT CODE) 

10. Where it says "D" write the number . (SCHOOL CODE) 

Note: Disregard any other coding that may appear on the sheet. 
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DURING THE ASSESSMENT (Continued) " ^ 

Students are now pr epared to begin the writing assessment . 

^^l' ^^^jnde 3, all students in a given classoon will have the same topic. At 
grade 8, however, students in a given classroom nay have different topics. 

NOTE: Four testing conditions are shown below. For the pilot test, it is vital 
that you, as test monitor, follow only the condition that is checked (X). 

„ , Oral reading with teacher-directP d orewritino . Read "Background" aloud to 

the class as students follow along. Direct them to do the three prewriting steps 
that follow the "Background." Invite a few students to share aloud some of their 
words and ideas from the three steps. Acknowledge these ideas positively Then 
read 'Directions" aloud to the class and tell students to begin their planning 
and writing. (Total time should be about five minutes.) 

Oral reading without teacher-dir ected prewriting . Read "Background" aloud 

to the class as students follow along. Pause. Then read "Directions" aloud to 
the class and tell students to begin their planning and writing. 

,r_- ?11gnt mding with teacher-directe d orewritino . Tell students to read 
Background silently. Allow time for this reading. Direct them to do the three 
prewriting steps that follow the "Background." Allow about three minutes for 
prewriting. Then tell students to read "Directions" silently and begin their 
planning and writing. (Total time should be about five minutes.) 

- Silent reading without teacher-dir ected prewriting . Tell students to read 
Background silently. Allow time for this reading. Then tell students to read 
Directions silently and begir? their planning and writing. 

Move around the room and respond to questions regarding directions. Do not 
provide spelling words, advice about writing, etc. 

Make three announcements about time during the assessment: 

-- Your time is half up. You have 15 minutes to go. 

-- You need to complete your writing in five minutes. 

Our time is up. Finish the sentence you are writing and close your packet. 

Collect the papers after a total elapsed time of 45 minutes. 

AFTER THE ASSESSMENT 

1. Fill out the one-page "Teacher Response Form," which accompanies these 
materials. Put the student papers and questionnaire in the mailback envelope. 
Give envelope to your school principal. 

2. As assessment results become available at your school --probably in early 
January, 1988--discuss these with your principal and teaching colleagues^- Watch 
for Utah Writing Project workshops on the teaching of writing--or call 750-2222 
for further information. 
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TEACHER RESPONSE FORM PILOT TEST. UTAH WRITING ASSESSMENi, 10/87 

DirecUonf formers") ^" ^^ ^"'^ ""^^ ^ of "General 

H*flht'*r^L!SlT/^ -f? """ment did you adainister? (See code number in upper 
right comer of writing prompt sheet used by students) 

4. Uhat difficulties did you have with test administration? 



5. What difficulties did students have in taking the test? 



the {Jj^^^^Westions would you make for improving the content (writing topic) of 



7. Uhat suggestions would you make for improving the test directions? 



8. What are some ways to simplify test administration from your point of view? 



Many thanks for your cooperation on the pilot test! 

ERIC 



APPENDIX D 
SAMPLE LETTER TO SUPERINTENDENTS 




111 



1'^ 



APPENDIX D; SAMPLE LETTER TO SUPERINTENDENTS 
UTAH STATE UNIVERSITY 

COLLEGE OF EDUCATION 
Department of Secondary Education 
Logan. Utah 84322-281 5 
(801)750-2222 



September 3, 1987 



Superintendent 

School 

Street 

City, State Zip 
Dear Superintendent: 

The Utah State Office of Education plans to a.' 3ss writing 
performance at grades 3 and 8 during 1988. This summer teachers met to 
study the Core Curriculum in English/language arts and to develop 
assessment items in writing that addressed its standards. 

We are asking for your cooperation in carrying out a pilot test of 
the writing assessment. Your district would be one of sixteen which would 
be involved in the pilot testing. We would like to use two schools in 
your district, in this case Washington Terrace School and Roy Jr High 
School with two different teachers and classes at grades 3 and 8. 

This pilot test will check the writing tasks, teacher and student 
directions, and scoring procedures for the Utah Writing Assessment, which 
will occur in 1988. The pilot test is being conducted by the Utah Writing 
Project (UWP) under the auspices of the Utah State Office of Education. 

We propose to use the principal at each school as the contact person 
for the test session which is tentatively scheduled for October 1 or 2. 
The testing will take no more than one class period to administer. 

We will share the results of the assessment with each district that 
participates in the pilot test. If you have any questions or do not want 
the above schools to participate in the study, please inform us 
immediately. We can be reached at 750-2222. Your support in this 
important effort is greatly appreciated. 

Sincerely, 



Charles R. Ouke 
Co-director, UWP 



William Strong 
Co-director, UWP 
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APPENDIX I: SAMPLE LETTER TO PRINCIPALS 




UTAH STATE UNIVERSITY 



COLLEGE OF EDUCATION 
Department of Secondary Education 
Logan. Utah 84322-281 5 
(801)750-2222 



September 3, 1987 



Principal 

School 

Street 

Town, City Zip 
Dear Principal : 

The Utah State Office of Education plans to assess writing performance at grades 
3 and 8 during 1988. Last summer teachers met to study the Core Curriculum in 
English/Language arts and to develop assessment items in writing that addressed 
its standards. 

Yours is one of thirty-two schools selected as a Utah Writing Assessment pilot 
test site. We are therefore asking your cooperation in a pilot test at grades 3 
and 8 on October 1 or 2 (your option on date). In return we will share the 
results of this pilot test with you. 

This pilot test will check writinq tasks, teacher and student directions, and 
scoring procedures for the Utah Writing Assessment, which will occur during 1988. 
It is conducted by the Utah Writing Project (UWP) under the auspices of the Utah 
State Office of Education. 

Your responsibilities as a site coordinator for the pilot test are: 

1. Obtain the cooperation of iyjo teachers at grade 3 or grade 8, whichever is 
appropriate for your school. (Note: Each teacher at grade 8 will assess 
only one class.) Phone in the number of test packets required by each 
teacher. 

2. Receive testing materials, look them over, and distribute to teachers. 

3. Serve as a communication link between UWP and teachers, handling any 
questions the teachers may have and communicating them to UWP. 

4. With the cooperation of teachers, monitor the administration of the test on 
October 1 or 2 and provide UWP with any suggestions for revised procedures. 

5. Collect testing materials and mail them back to UWP. 

We welcome your leadership in this statewide effort. By September 11, please 
phone the Utah Writing Project at 750-2222 to indicate whether you are willing to 
serve as a site coordinator and, if so, the test packets required. 



Sincerely, 



Charles R. Duke, 
Co-Director, UWP 



William Strong, 
Co-Director, UWP 
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UTAH STATE UNIVERSITY 

COLLfCE OF EDUCATION 
Oep«rtment ot Secondary Education 
Logdn Utah 84327-281 S 
(601/750-2222 

MEMORANDUM 



TO: Principals participating in Writing Assessment 

FROH: Charles Duke and William Strong, co-directors UWP 

SUBJECT: Test Packets and Test Administration 

DATE: September 16, 1987 



We appreciate your serving as a site coordinator in the pilot test of th 
Utah Writing Assessment. Enclosed in this mailing are test packets for 
the two teachers participating in the pilot test. Note that the packe:s 
have numbers on them; these should correspond to the number of students 
you identified for us in each class. 

With the teachers, review the enclosed "Directions for Teachers" that are 
a part of each test packet; a copy is provided for you as well. 

We ask that persons participating in the pilot test help us with security 
by not photocopying the assessment or discussing its contents with 
colleagues or students. We also ask that you be sure to collect all 
tests, even extras, and return them to us. You and your teachers may be 
assured that we will maintain '^trict confidentiality regarding teacher 
identities i. our report of test result?. 

On the day of the test, please arrange to .nonitor the administration of 
the test in at least one classroom. Check to see how clear the directions 
are, whether the task seems appropriate, and what, if any, difficulties 
students appear to have. We'd appreciate your sending along your 
observations when you return the tests. 

After the tests have been administered, collect from each teacher the 
tests and written responses plus the "Teacher Response Form; see that each 
tea ..r places his or her tests and form in the envelope in which the 
tests came; then place both envelopes 'n the self-addressed, stamped 
envelope you received and send it to us immediately. In case you misplace 
the envelope, materials should be returned to: 

Utah Writing Project Pilot Test 
Departiflfent of Secondary Education 
Utah State University 
Logan, UT 8-322-2815. 

If you have any questions, feel free to call us at 75C-2222. 
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APPENDIX F: REPRESENTATIVE COMMENTS FROM TEACHER SURVEY 



Grade 3 

1. What difficulties did you have with test administr :tion 

a. I didn't make clear that the planning page was not part of the start of 
the story 

b. Need to state the type of paper to be used. 

c. Before this assignment was given to me--about a week-- I had been 
teaching the children how to "map" story events. Much of that 
experience flowed through these papers. They still do not have that 
skill, but it did influence their planning. 

d. Not having the test packet available until morning of test date 

e. Explaining some of the language used in the background and prompt 

f. My students were not accustomed to miking a plan before writing so this 
part was a little difficult for the:.i. 

g. Students wanted help with spelling. 

h. For some students it was too much time; all finished before 30 
minutes. 

1. We are on the enhanced schedule and I have basics in the morning--is 

there a difference between morning and afternoon? 
j. Students should have four or five practice sessions before actual test; 

the sample exercises would help establish procedures, 
k. Some students asked if they could erase a mistake; the directions state 

they could cross out a word but really didn't make clear if erasing was 

allowed. ^ 

1. Some students had trouble with directions and wanted help; I told them 

words like audience and experience, 
m. It was hard to give instructions in just 15 minutes. I wanted to give 

more instruction than what was written n instruction sheet 
n. Not interacting with students to develop ideas. 

2. What difficulties did students have in taking the test? 

a. Lined paper was too large; third graders can write on regular paper 

b. Need pre-writing time--3rd graders need shorter writing time 

c. Being asked to do thinking and planning 

d. Proofreading was not very evident 

e. Third graders, toward the end of the year, have a pretty good idea of 
what we mean by main idea and support ideas. At this point in time, 
though, paragraphs are hard to understand. 

f. Students had hard time understanding questions 1-3 under background 
(prewriting); words like "unusual" and "describe" are hard for 3rd. 

g. They would often write on the back side of page 

h. They wanted to know how to spell words even after I told them to spell 
them as best they could 

i. They had difficulties mainly in the "Planning" stage. Soi.'e actually 
wrote instead of jotting things down. 

j. We haven't got to creative writing until now so the students were 
having a hard time writing sentences and putting thoughts into writing, 
i. No major problems but some wanted to write only one sentence stories 
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What suggestions would you ...dke for imoroving the content (writing topic) 
of the test? 

a. Good one--all children can identify a place 

b. Students enjoyed the assignment topic on their favorite place; it was 
fairly easy for them to organize their thoughts 

c. Use words like "special" or "a day you enjoyed or didn't like"--words 
third graders understand now 

d. My student teacher had just had to reprimand some students for playing 
kissing tag at noon recess and because this was the most recent thing 
on their minds, they wrote about tha. instead of a real unusual day. We 
probably should have waited until this was not such a big issue. 

e. Some of the directions were useful for planning; they could have been 
repeated again for the actual writing 

f. I liked the writing topic because there wc/e ways to practice it 
without actually using the topic 

g. The word "possession" was not readily understood by students. 

n. Some students seemed to have difficulty thinking of real events. I 
tried to steer them away from make believe things; perhaps at this age 
make believe things might be better? 

i. Every child knew of something they wanted o tell about. 

What suggestions woulct you make for improving the test directions? 

a. Perhaps less indepen(»%.it reading 

b. Were covered in a lucid manner. 

c. There has to be a little more teacher direction in o-al directed 
reading without teacher directed prewriting. Student background 
information should also be on the teacher directions 

d. Is there c special reason for not having students erase? 

e. Include the option of real or imaginary experiences. 

J would not put the directions on the same page allotted for 
planning. 

g. Include that a story needs a beginnning, middle, and end. 

h. In the directions section it might be helpful if the words following 

The reader wants to kncv#..." v;cre in a list for easy reference instead 
of sente 'ces. 

i. Allow time for questions. 

What are some way.i to simplify test administration? 

a. You've got as simple as possible; don't simplify further. 

b. Lines on writing paper too wide; use regular paper with lines 

c. Nothing, it was fun. 

d. Simplify some of the language for third grade 

e. It was a good idea in my class for students to have an extra sharpened 
pencil at their desks 

f. I taped a paper on which was their 4 digit coda to their desks 

g. Quite pleased with administration 

h. The test administration would go quicker if the teacher filled out the 
code numbers in advance; then less time would have to be spent on 
directions 

i. If we were control, we would do it that way; if not, they need some 
preparation ahead. 
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1. What difficulties did you have with test administration? 

a. None 

b. Many students forgot #2 pencils or pens because of the three day break 
between test announcement and administration; test was easy to 
administer. 

c. No teacher's copy of student writing directions; had to look over 
child's shoulder to read aloud directions. 

d. Receiving packets earlier than week of administration would be helpful 

e. Total writing time was 30-32 minutes after writing background question- 
and answers for 5 minutes 

f. I wasn't sure how the timo was to be divided; it wasn't clear when I 
should start watching the clock 

g. My instruction booklet was not marked as to which test condition I was 
to use so I guessed and hope I did right 

h. Did you want their prewriting? 

i. Students who wrote letters wanted to know how to sign it if it was to 
remain anonymous. The first one to ask suggested "an eighth grade 
student" so that it was. 

j. The students were not happy to write. 

k. We had to get some eighth graders from other v,lasses to fill our 
complement and some teachers didn't like to have their classes 
interrupted for that purpose; we had 35 students in a room built for 
25! We had Homecoming prior to test administration day; home-coming 
causes students to forget what school is all about; we had some 
discipline problems at the start of the test. 

2. What difficulties did students have taking the test? 

a. No real problems 

b. Most students were able to do the prewriting but coming up with ideas 
was in many cases hard. 

c. Didn't understand the Rock and Roll instructions; didn't know if Kelly 
was male or female. The snake piece was a long one. 

d. Wondered about margins; whether they should 3 pages since that number 
was provided; some thought they should write for 30 minutes. 

e. Students might have had a bit too much time for the letter and the 
possession; the speech and the snakes seemed harder tasks. 

f. .'id not ]*\e idea of crossing out mistakes 

g. Questions about titles--should they have one? 

h. Students want directions read to them; they uon't want to take 
responsibility for their own writing. 

i. Should names be put on the papers? 

j. Many of our Navajo students don't read well enough to understand 
written directions. I had to explain the directions without giving away 
the test so some students got started late on actual drafting. We've 
only been teaching writing process for two years 

k. Students seemed to have irore problems with understanding and doing-- 
8-3-C, 8-4-4. 

1. Students wnre frustrated because they are used to some kind of 
prewriting; they were also concerned that they would not have time for 
revision. 
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3. What suggestions would you make for improving the content (writing topic) 
of the test? 

a. Students seemed to be interested; no changes 

b. I like the fact it's a personal topic that each one of them can relate 
to. 

c. Good tf^iiiC'-perhaps simplify Rock and Roll one; fewer notes for snakes 
one to be incorporated into paragraph 

d. The letter writing and the favorite possession draw from a student's 
personal experience and opinions and can, therefore, be done without 
other specific knowledge or skill. The other two, the speech and the 
newspaper article call for a sense of audience on the part of the 
writer and a familiarity wUh a more objective style of writing. That 
makes them different kinds of tasks. 

e. Some kids won't or can't handle any topic. The topics seemed like good 
ones. The fewer directions the better. 

f. The letter worked; students understood with little explanation; the 
newspaper prompt was confusing. The kids weren't sure which snake to 
write about. Maybe if you had only had notes on one snake then the kids 
could jump right in. 

g. 8-4-4 needs to be more relevant; 8-3-8 needs clarity on what they're 
writing about. 

h. Personal topics are good motivators. 

i. Topics were ones my students do not ordinarily write on. 

j. I think few eighth graders think deeply enough to react in a logical 
and intelligent way to the issue of rock music; the topics did not seem 
'Bqui table; the students who wrote the newspaper article worked much 
harder than the others; they wanted to trade and write a friendly 
letter. 

4. What suggestions would you make for improving the test directions? 

a. None. I really liked the statement in purpose of the assessment to see 
how well Utah students write . 

b. Outline how the time is to be spent for the 15 min. segment. 

c. They were fine with me; just tell me what to do and Til do it. 

d. Was it okay to erase? 

e. They're great; maybe specify point of view if you want to in the rock 
and roll prompt. 

f. None except maybe giving students an example 

g. Ask for specific details and give an example; impt. for this level 

h. Prewriting discussion and activities are always a part of my teaching 
of writing. This exercise seemed cold; also, my students did not feel 
comfortable without tools of writing (thesaurus, dictionary, etc.) They 
are also accustomed to feedback. 

i. They worked well for us! 

j. Students asked if it mattered how short it was 
k. There was no x marked for monitor on condition 

5. What are some ways to simplify test administration? 

a. I thought the instructions quite complete. 

b. I think the test is already easy to administer. 

c. My students were more interested in the "why" a.id "who" of this test 
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d. Keep it simple--which you did! 

e. Direc-.ions were clear-especially 1-10 before assessment p. 1 

f. The only problems we had were of our own making; if we do it aqain next 
year we'll be better prepared. 

g. Terminology was a problem for my kids in truly understanding. Navajo 
students students have difficulty in areas of relevancy; what's 
relevant to a white student may not be to a Native American. 

h. Have them do their best by prewriting, editing, final draft 

1. Unless you change the nature of the test, I don't feel the 
administration needs simplification; students had few questions and 
seemed to have adequate time. 
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APPENDIX G: DIRECTIONS FOR SCORING 



UWP Writing Assessment Project 
Holistic Scoring 



Directions to Readers: 

The papers you are about to read were written by Utah students Our 
efforts today are to score the papers, using a rubric designed specifically for 
the prompt and the student population. You will receive this rubric separately 
and will practice with it until you are familiar with its categories and until 
the group of readers as a whole has developed consistency in its use. 

If you are unfamiliar with holistic scoring, you should be aware that 
research has shown that the process is as valid and reliable as the more 
traditional--and slower—analytical scoring of writing samples. In holistic 
scoring, you are to use a "whole impression" of the work which reflects the 
various characteristics of the scoring rubric to determine where the piece of 
writing falls on a given scale. No single item or characteristic is 
necessarily more important than another in this system; in fact, holistic 
scoring is not designed to be diagnostic but is more useful in showing large 
scale changes within a given population and for placing students in appropriate 
courses or programs. Our intent in this particular evaluation project i< to 
determine what the level of skill currently is among 3rd and 8th grade students 
in Utah. 

Since we will be reading a large number of papers and accuracy is of 
great importance, we ask that you observe the following procedures carefully: 
If you have questions at any time, feel free to consult the Head Reader. 

Before the Official Reading 

1. Arrange your materials carefully so that you can refer to them easily 
and efficiently; you will have the following as resources: 

a. copy of the prompt used for the writing 

b. copy of the rubric--you .nay want to unstaple the pages for 
easiei reference 

c. copies of "range" markers for each category on the rubric (these 
will be provided during practice) 

d. two marked spaces in front of you, one of these will be for 
papers you are to score; the other will be for papers you have 
scored and which are ready for pick-up 

e. pencil for recording scores 

2. Feel free to ask questions during the practice session. We need to 
train quickly, but we also need to understand clearly what we are 
doing; the Head Reader will attempt to answer your questions. Listen 
carefully to the discussion, apply the rubric carefully, and observe 
how your scores compare with those of the groun. The basic purpose 
of the training session is to reach consensus on the application of 
the rubric to the writing sa.iples. 



123 



During the Official Readlnn 



1. Resist the temptation to read an excerpt to another reader; there 
win be ample time to swap choice bits with each other during the 
breaks. 

2. Give to the Head Reader any illegible papers which you feel are 
beyond any effort at reading; handwriting on some of the papers will 
not be good but try not to let t.iat influence you unduly; remember 
that students had little time to consider overall neatness. 

3. Read as quickly =»s possible and try to score /our first impression; 
if you find yourself re-reading the papers, you probably have lost 
confidence in your memory of the rubric; review the categories and 
then start reading again. Remember that the goal is overall 
impression. Isolating one part over another tends to reduce your 
effectiveness as a reader. 

4. Record the appropriate score in the location as indicated by the 
following code: 

a. NO COLOR ON FIRST PAGE--record score on the bagk of the LAST 
PAGE in the CENTER of that page; be sure the nuR.ber is formed 
clearly. 

b. COLOR ON FIRST PAGE- -record score at the too rioht-hand corner 
of the FIRST PAGE; be certain the number is formed clearly. 

5. Place all scored papers on the "Out" pile in front of you; these 
papers will be picked up periodically. The Head Reader may read some 
of these at random, simply to insure that all readers are using the 
rubric appropriately; on occasion, the Head Reader may discuss some 
of your scores with you to make certain that you and he or she have a 
consensus on the scores. Remember that the succejs of this kind of 
evaluation depends upon consistency in scoring. 

Other Considerations 

1. We will take periodic breaks--reader fatigue is a definite factor in 
these sessions--so that you can obtain refreshments, visit the rest 
rooms, get some fresh air. These breaks, however, will be short-- 
seldom more than ten minutes except for a lunch break. Please be 
prompt in starting reading again as soon as the signal is given. 

2. You will find reading these papers provides you with some insights 
into how today's students think and also how they respond to a 

•writing situation. You may or nay not agree with their stands on an 
issue or their choices of subject; Your focus at all times, however, 
must be on evaluating the papers against the rubric, not against your 
own personal beliefs or expectations. 

3. We appreciate the time you are giving us for the scoring session. If 
it is necessary for you to leave before the time allotted for s'^oring 
is over, please inform the Head Reader so that the time you did spend 
can be recorded. 
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APPENDIX H: MEAN COMPARISON 
GRADE 3 

Comparison of Means 



PROMPT 


CONDITTOM 




MALES FEMALES 


D-VALUE 


Place 


Oral w/ 
Prewriting 


Mean 

Std 

N 


3.27 3.86 
1.22 1.71 
66 50 


0. 03* 


Oral w/o 
Prewriting 


Mean 

Std 

N 


3.41 3.86 
1.75 1.87 
53 58 


0.20 


Possession 


Oral w/ 
Prewriting 


Mean 

Std 

N 


3.47 4.29 
1.36 1.66 
61 51 


0.01** 


Oral w/o 
Prewriting 


Mean 

Std 

N 


3.13 4.41 
1.46 1.57 
62 61 


0. 00** 


Day 


Oral w/ 
Prewriting 


Mean 

Std 

N 


3.30 2.77 
1.50 1.46 
60 39 


0. 08 


Oral w/o 
Prewriting 


Mean 

Std 

N 


3.63 4.66 
2.16 2.12 
52 50 


0.02* 


Scary 


Oral w/ 
Prewriting 


Mean 

Std 

N 


3.47 3.59 
1.77 1.51 
57 42 


0.72 


Oral w/o 
Prewriting 


Mean 

Std 

N 


2.90 3.31 
1.55 2.08 
50 49 


0.27 
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GRADE 8 

Comparison of Means 



PROJfPT 


CONDITION 




MAT.KS FEMALES 


D-VALUE 


Letter 


Silent w/o 
Prewriting 


Mean 

Std 

N 


3.16 4.25 
1.38 1.56 
69 91 


0.00** 


Possession 


Oral w/ 
Prewriting 


Mean 

Std 

N 


3.70 4.84 
1.44 1.79 
57 57 


0.00** 


Oral w/o 
Prewriting 


Mean 

Std 

N 


3.03 3.79 
1.42 1.51 
32 38 




Silent w/ 
Prewriting 


Mean 

Std 

N 


4.33 5.35 
1.41 1.57 
33 46 


0.00** 


Silent w/o 
Prewriting 


Mean 

Std 

N 


3.79 4.27 
1.46 1.48 
62 88 


0.05* 


Music 


Silent w/o 
Prewriting 


Mean 

Std 

N 


2.97 4.07 
1.38 1.67 
60 87 


0.00** 


Snakes 


Silent w/o 
Prewriting 


Mean 

Std 

N 


4.02 4.67 
1.97 1.60 
60 81 


0.03* 



o 
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APPENDIX I: THIRD READINGS 

THIRD READINGS 



GRADE 3: NUMBER OF THIRD READINGS BY PROMPT AND CONDITION 



PRQMEI CONDITION TOTALS 

ORAL W/ (1) ORAL W/0 2) 

PREWRITING PREWRITING 



1 

Place (1) 1 (.99*) 


5 


(.95*) 


6 


(.97*) 


Possession (2) 5 (.95*) 


12 


(.90*) 


17 


(.93*) 


Day (3) 4 (.96*) 


2 


(.98*) 


6 


(.97*) 


Scary (4) 5 ^..95*) 


4 


(.96*) 


9 


(.95*) 


Totals 15 (.96*) 


23 


(.94*) 


38 


(.96*) 


*denotes inter-rate agreement 











GRADE 8: NUMBER OF THIRD RtADINGS BY PROMPT AND CONDITION 

PROMPT CONDITION TOTALS 

(i) (2) (3) (4) 

ORAL W/ ORAL W/O SILENT W/ SILENT W/O 

PREWRITING PREWRITING PREWRITING PREWRITING 



Letter (5) 5 (.37*) 5 (97*) 

Possession 10 (.91*) 4 (.91*; 2 (.97*) 10 (.93*) 21 (.93*) 

(6) 

Music (7) 14 (.90*) 14 (.90*) 

Snakes (8) 

Totals 10 (.91*) 4 (.91*) 2 (.97*) 29 (.93*) 40 (.93*) 
*denotes inter- rater agreement 
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APPENDIX J ERROR ANALYSES 



THIRD GRADE (SCOREPOINT 8): 1312 total 


words; mean length = 131.2 


I. Punctuation (sentence end stop) 


Errors 


Error Rate 


A. Fragment 


7 


.005 


B. Comma splice/fused sentence 


Z4 


.018 


C. On-and-on sentence 


2 


.002 


II. Usage 






A. Subject/verb agreement 


2 


.002 


B. Capital ization 


17 


.013 


III. Spelling 






M. Homonyms/contractions 


8 


.006 


B. General spelling 


91 


.069 




THIRD GRADE (SCOREPOINT 6): 915 total words; 


mean length > 91.5 


I. Punctuation (sentence end stop) 


Errors 


Error Rate 


A. Fragment 


7 


.008 


B. Comma splice/fused sentence 


16 


.017 


C. On-and-on sentence 


7 


.008 


II. Usage 






A. Subject/verb agreement 


2 


.002 


B. Capitalization 


20 


.022 


III. Spelling 






A- Homonyms/contractions 


7 


.008 


B. General spelling 


75 


.082 
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THIRD GRADE (SCOREPOINT 4): 1055 


total words 


; mean length = 1055 


I. 


Punctuation (sentence end stop) 


'"r*'ors 


Error Rate 




A. Fragment 


3 


.003 




D. Comma splice/fused sentence 


24 


.023 




C. On-and-on sentence 


11 


.010 


II. 


Usage 








A. Subject/verb agreement 


2 


.002 




B. Capitalization 


16 


.015 


III. 


Spelling 








A. Homonyms/contractions 


10 


.009 




B. General spelling 


95 


.090 



THIRD GRADE (SCOREPOINT 2): 643 total words; mean length = 64.3 v^ords 


I. 


Punctuation (sentence end stop) 


Errors 


Error Rate 




A. Fragn-ont 


3 


.005 




B. Comma splice/fused sentence 


22 


.034 




C. On-and-on sentence 


8 


.012 


II. 


Usage 








A. Subject/verb agreement 


0 


.000 




B. Capitalization 


27 


.042 


III. 


Spelling 








A. Homonyms/contractions 


2 


.003 




B. General spelling 


94 


.146 
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EIGHTH GR/iDE (SCOREPOINT 8): 2565 total words; 


mean length = 256.5 words 


I. Punctuation (sentence end stop) 


Errors 


Error Rate 


A. Fragment 


2 


.001 


B. Comma splice/ fused sentence 


26 


.010 


II. Usage 






A. Subject/verb agreement 


2 


.001 


B. Capital IZdtlon 


5 


.002 


III. Spelling 






A. Homoiyms/contractlons 


2 


.001 


B. General spelling 


25 


.010 





EIGHTH GRADE (SCOREPOINT 6): 3132 


total words; 


mean length = 313.2 words 


I. 


Punctuation (sentence end stop) 


Errors 


Error Rate 




A. Fragment 


9 


.003 




B. Comma splice/fused sentence 


36 


Oil 


II. 


Usage 








A. Subject/verb agreement 


2 


.001 




B. Capitalization 


7 


.002 


III. 


Spelling 








A. Homonyms/contractions 


12 


.004 




B. General spelling 


15 


.016 
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EIGHTH GRADE (SCOREPOINT 4): 2208 total words; mean length = 220.8 words 


I. Punctuation (sentence end stop) 


E rrors 


Error Rate 


A. Fragment 

D. Lomima spi ice/Tuseo sentence 


7 
30 


.003 
.014 


11. Usage 






A. Subject/verb agreement 

B. Capitalization 


0 
4 


.000 
.002 


ilL Spelling 






A, Homonyms/contractions 

B. General spelling 


22 
87 


.010 
.039 





EIGHTH GRADE (SCOREPOINT 2): 1372 


total words; mean length » 137 words i 


I. 


Punctuation (sentence end stop) 


Errors 


1 

Error R>te j 




A. Fragment 

B. Comma splice/ fused sentence 


1 

17 


.001 
.012 


II. 


Usage 








A. Subject/verb agreement 

B. Capitalization 


1 

12 


.001 
.009 


III. 


Spelling 








A. Homonyms/contractions 

B. General spelling 


19 
38 


.014 

.028 1 
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APPENDIX K: WRITING PROMPT MATERIALS 



UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



3 - 



1 - 1 



A 



B 



C 



0 



Having Fun at a Favorite Place 



Background 



Most people have had a fun experience at a favorite place. For you, it might be 
building a hut in the backyard, playing ball in a neighborhood park, roasting 
marshmallows at a campground, or riding the roller coaster at Disneyland. Think 
about what you really l^ke to do at your favorite place. 

1. Write down a favorite place where you have fun. 



2. What makes this place fun for you? Write this down. 



3. What words describe your favorite place? List them. 



Use this page to plan your writing. Beginning on the next page, tell a story 
about a time when you had fun at your favorite place. The reader would like to 
know what this place is like, so describe it. Tell what you did, how you felt, 
and why it was fun to you. 



Directions 
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UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



3-1-2 



A B D 



Having Fm at a Favorite Place 

Background 

Most, people have had a fun experience at a favorite place. For you, it might be 
building a hut in the backyard, playing ball in a neighborhood park, resting 
marshmallows at a campground, or riding the roller coaster at Disneyland. Think 
about what you really like to do at your favorite place. Use this page to plan 
your writing. 

Directions 

Beginning on the next page, tell a story about a time when you had fun at your 
favorite place. The reader would like to know what this place is like, so 
describe it. Tell what . lu did, how you felt, and why it was fun to you. 
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UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



3-2-2 



A B C D 



How I Got My Favorite Possession 

Background 

Like most people, you probably own something that is very special. It might be a 
pet, a toy, a book, or something you wear. Maybe this possession is special 
because of the person who gave it to you or because you earned it yourself. 
Think about the day you got a favorite possession and v^hy you like it. Use this 
page to plan your writing. 

Directions 

Beginning on the next page, tell a story about when you got your favorite 
possession. The reader would like to know what it looks like, so describe it. 
Tell what happened when you got it, how you felt, and why it is important to you. 
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UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 

A B C D 



How I Got Ny Favorite Possession 

Buckqround 

Like most people, you probably own something that is very special. It mi^ht be a 
pet, a toy, a book, or something you wear. Maybe this possession is "special 
because of the person who gave it to you or because you earned it yourself. 
Think about the day you got a favorite possession and why you like it. 

1. Write down your favorite possession. 



2. What makes this possession special to you? Write this down. 



3. What words describe your favorite possession? List them. 



Directions 

Use this page to plan your writing. Then, beginning on the next page, tell a 
story about the time you received vour favorite possession. The reader would 
like to know what it looks like, so describe it. Tell what happened when you got 
it, how you felt, and why it is important to you. 
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UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



3-3-1 



A 



B 



C 



D 



An Unusual Day at School 



Backqroun(;i 



All of us have had an unusual day at school. For you, it might have been a a 
dress-up day, a fight on the school playground, or a substitute teacher. Maybe 
you won an award or celebrated a holiday. Think about a day at school that 
really stands out in your mind. Use this page to plan your writing. 

1. Write down an unusual day at school. 



2. What made this day so unusual or different? Write this down. 



3. What words describe this unusual day? List them. 



Beginning on the next page, tell a story about an unusual day at school. The 
reader would like to know what this day was like, so describe it. Tell what 
happened, how you felt, and why you remember this unusual day at school. 



Directions 
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UTAH WRITING ASSESSMENT PILOT TEST 10/87 



3-3-2 



A B c D 



An Unusual Day at School 

Background 

All of us have had an unusual day at school. For you, it might have been a 
dress-up day, a fight on the school playground, or a substitute teacher. Maybe 
you won an award or celebrated a holiday. Think about a day at school that 
really stands out in your mind. Use this page to plan your writing. 

Directions 

Beginning on the next page, tell a story about an unusual day at school. The 
reader would like to know what this day was like, so describe it. Tell what 
happened, how you felt, and why you remember this unusual day at school. 
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UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



A Scary Experience 

Background 

Most people have had a scary experience. Maybe they were frightened by a snake, 
or big dog or even a bee. Maybe they were scared by a bully at school. Maybe 
they were lost in a mall or frightened by an audience. Think about a scary 
experience you have had and why you still remember it. Use this page to plan 
your writing. 

1. Write down a place where you were scared. 



2. What made this experience scary? Write this down. 



3. What words describe this scary experience? List them. 



Directions 



Use this page to plan your writing. Beginning on the next page, tell a story 
about a time you were scared. The reader wants to know where you were and what 
made you feel scared. Tell what happened, how you felt, and why you still 
remember this scary experience. 
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3-4-2 

UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



A Scary Experience 

Background 

Most people have had a scary experience. Maybe they were frightened by a snakt, 
or big dog or even a bee. Maybe they were scared by a bully at school. Maybe 
they were lost in a mall or frightened by an audience. Think about a scarv 
experience you have had and why you still remember it. 

Directions 

Use this page to plan your writing. Beginning on the next page, tell a story 
about a time you were scared. The reader wants to know vhere you were and what 
made you feel scared. Tell what happened, how you felt, and why you still 
remember this scary experience. 
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UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



Background 

Assume that your principal has received the following letter with a note asking 
that It be passed on to an eighth grader for a response. The letter has now 
been given to you. 

Dear Eighth Grader, 

I will be moving to your area and will be 'n the eighth grade 
at your school next year. Please write me a letter telling me 
about your school . 

I would especially like to know about teachers, classes, 
activities, and anything else that will help me fit in to the 

scliool . 

Write soon. 

Sincerely, 
Kelly Green 

Directions 

Write a letter to Kelly that tells about your school. Use your own mailing 
address for the return address so that if Kelly wants to write for more 
information, you will receive the letter. The principal has Kelly's address ai u 
win see that your letter is sent. Since Kelly has never visited your school, 
provide as much helpful information in your letter as you can. Be certain to 
include the information Kelly has requested. Write only on one side of the 
page. Begin your letter on the next page. 
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UTAH WRITING ASSESSMENT - PILOT TEST 10/87 



A B C D 



How I Got Ny Favorite Possession 

Background 

Like most people, you probably own something that is very 
special. It might be a pet, a toy, a book, or something you 
wear. Maybe this possession is special because of the person who 
gave it to you or because you earned it yourself. Think about 
the day you got a favorite possession and why you like it. Use 
this page to plan your writing. 

Directions 

Beginning on the next page, tell a story about when you got your 
favorite possession. The reader would like to know what it looks 
like, so describe it. Tell what happened when you got it, how 
you felt, and wK^ it is impo tant to you. 
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UTAH WRITING ASSESSMENT -- PILOT TEST 10/87 



HoM I Got Ny Favorite Possession 

Backgr o und 

Like most people, you probably own something that is very 
special. !t might be a pet, a toy, a book, or something you 
wear. Maybe this possession is special because of the person who 
gave it to you or because you earned it yourself. Think about 
the day you got a favorite possession and why you i;ke it. Use 
this page to plan your writing. 

1. Write down your favorite possession. 



2. What makes this possession special to you? Write this down. 



3. What words describe your favorite possession? List them. 



Directions 

Beginning on the next page, tell a story about the time you 
received your favorite possession. The reader would like to know 
what it iGoks like, so describe it. Tell what happened when you 
got it, how you felt, and why it is important to you. 
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UTAH WRITING ASSESSMENT - PILOT TEST 10/87 



Background 

Parents and students at your school are divided in their opinions about the 
effect of rock music on students. Some want to eliminate all rock music on 
radio stations and at school dances. Others don't view rock music as a 
problem. 

Because the issue is one that has caused great interest among people in your 
community, the local radio station has scheduled a public meeting where oeoDle 
can speak about this issue. v^^v*^ 



Decide which side of this issue you favor. Write a speech designed to persuade 
others to agree with you. Your speech should include personal experience 
and/or other evidence to support your position. Your audience will consist of 
bot^ parents and other students. 
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UTAH WRITING ASSESSMENT - PILOT TEST 10/87 



8-4-4 



Background 

John Burton, an expert on snakes, presented an assembly in your school 
auditorium. He brought with him a live rattlesnake, an Indian python, and a 
king cobra. You are on the newspaper staff at your school and have been asked 
to write a news story about the assembly. 

Directions 

Write a news story about the assembly for your school newspaper. Assume you 
have taken the following notes. Select the information that you can use to 
write an effective news story. Remember that the first paragraph of a news 
story contains the 5 Ws (who, what, when, where, and why.) Organize your 
remaining information so that it is interesting and easy for your readers to 
understand. 



Diamond Rattlesnake 
iiamed Elvis 
has long hollow fangs 
is most dangerous rattler 

in U.S. 
rests with body coiled 
has hollow teeth like 

hypodermic needles 
raises upper third of body 

into S-shape when angry 
stabs prey 

warns of danger with rattle 
swallows food whole 
lives in U.S. 
strikes full length of 

S-shape 
gives birth to live babies 



named Crackin 
8 ft. from India 
held by 14 members of 

audience 
lays eggs 
is sluggish, lazy 
Is found near water 
is not poisonous 
kills by constricting 



YOUR NOTES 

Kino Cobra 

named Tinkerbell 

is 3 ft. snake from India 

strikes slowly 

attacks without being angered 
is hooded 

bites and chews with short fangs 
as venom runs into wound 

has deadly bite with no antidote 

was shedding and was irritable 
and nervous 

doesn^t move to snake charmers* 
music. 

Sways to charmers* movement 
king cobra kills 25,000 Indian 

people per year 
large king cobras have enough 

venom to Kill hundreds of 

peopl 1 



large pythons kill goats, 

pigs, and deer 
python in Malaysia ate a 14 

year old boy 
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